我正在尝试使用 spacy v3.0 训练我的数据，显然 nlp.update 不接受任何元组。这是一段代码：
import spacy
import random
import json
nlp = spacy.blank("en")
ner = nlp.create_pipe("ner")
nlp.add_pipe('ner')
ner.add_label("label")
# Start the training
nlp.begin_training()
# Loop for 40 iterations
for itn in range(40):
# Shuffle the training data
random.shuffle(TRAINING_DATA)
losses = {}
# Batch the examples and iterate over them
for batch in spacy.util.minibatch(TRAINING_DATA, size=2):
texts = [text for text, entities in batch]
annotations = [entities for text, entities in batch]
# Update the model
nlp.update(texts, annotations, losses=losses, drop=0.3)
print(losses)
我收到错误
ValueError Traceback (most recent call last)
<ipython-input-79-27d69961629b> in <module>
18 annotations = [entities for text, entities in batch]
19 # Update the model
---> 20 nlp.update(texts, annotations, losses=losses, drop=0.3)
21 print(losses)
~\Anaconda3\lib\site-packages\spacy\language.py in update(self, examples, _, drop, sgd, losses, component_cfg, exclude)
1086 """
1087 if _ is not None:
-> 1088 raise ValueError(Errors.E989)
1089 if losses is None:
1090 losses = {}
ValueError: [E989] `nlp.update()` was called with two positional arguments. This may be due to a backwards-incompatible change to the format of the training data in spaCy 3.0 onwards. The 'update' function should now be called with a batch of Example objects, instead of `(text, annotation)` tuples.
我设置了火车数据格式：
TRAINING_DATA = []
for entry in labeled_data:
entities = []
for e in entry['labels']:
entities.append((e[0], e[1],e[2]))
spacy_entry = (entry['text'], {"entities": entities})
TRAINING_DATA.append(spacy_entry)
我的列车数据如下所示：
[('Part List', {'entities': []}), ('pending', {'entities': []}), ('3D Printing', {'entities': [(0, 11, 'Process')]}), ('Recommended to use a FDM 3D printer with PLA material.', {'entities': [(25, 36, 'Process'), (41, 44, 'Material')]}), ('', {'entities': []}), ('No need supports or rafts.', {'entities': []}), ('Resolution: 0.20mm', {'entities': []}), ('Fill density 20%', {'entities': []}), ('As follows from the analysis, part of the project is devoted to 3D', {'entities': [(64, 66, 'Process')]}), ('printing, as all static components were created using 3D modelling and', {'entities': [(54, 66, 'Process')]}), ('subsequent printing.', {'entities': []}), ('', {'entities': []}), ('In our project, we created several versions of the', {'entities': []}), ('model during modelling, which we will describe and document in the', {'entities': []}), ('following subchapters. As a tool for 3D modelling, we used the Sketchup', {'entities': [(37, 49, 'Process')]}), ('Make tool, version from 2017. The main reason was the high degree of', {'entities': []}), ('intuitiveness and simplicity of the tool, as we had not encountered 3D', {'entities': [(68, 70, 'Process')]}), ('modelling before and needed a relatively flexible and efficient tool to', {'entities': []}), ('guarantee the desired result. with zero previous experience.', {'entities': []}), ('In this version, which is shown in the figures Figure 13 - Version no. 2 side view and Figure 24 - Version no. 2 - front view, for the first time, the specific dimensions of the infuser were clarified and', {'entities': []}), ('modelled. The details of the lower servo attachment, the cable hole in', {'entities': []}), ('the main mast, the winding cylinder mounting, the protrusion on the', {'entities': [(36, 44, 'Process')]}), ('winding cylinder for holding the tea bag, the preparation for fitting', {'entities': []}), ('the wooden and aluminium plate and the shape of the cylinder end that', {'entities': [(15, 25, 'Material')]}), ('exactly fit the servo were also reworked.', {'entities': []}), ('After the creation of this', {'entities': []}), ('version of the model, this model was subsequently officially consulted', {'entities': []}), ('and commented on for the first time.', {'entities': []}), ('In this version, which is shown in the figures Figure 13 - Version no. 2 side view and Figure 24 - Version no. 2 - front view, for the first time, the specific dimensions of the infuser were clarified and', {'entities': []}), ('modelled. The details of the lower servo attachment, the cable hole in', {'entities': []}), ('the main mast, the winding cylinder mounting, the protrusion on the', {'entities': [(36, 44, 'Process')]})]
作为新贡献者，我非常感谢您的帮助。非常感谢！
您没有提供您的
TRAIN_DATAfrom spacy.training.example import Example
for batch in spacy.util.minibatch(TRAINING_DATA, size=2):
for text, annotations in batch:
# create Example
doc = nlp.make_doc(text)
example = Example.from_dict(doc, annotations)
# Update the model
nlp.update([example], losses=losses, drop=0.3)
for batch in batches:
texts, annotations = zip(*batch)
example = []
# Update the model with iterating each text
for i in range(len(texts)):
doc = nlp.make_doc(texts[i])
example.append(Example.from_dict(doc, annotations[i]))
# Update the model
nlp.update(example, drop=0.5, losses=losses)
此代码在 Spacy 3 上成功运行。 请注意，这里我有一个字符串元组，如果您只想使用字符串，则不需要使用 for 循环。
从 spaCy 3.0 版本开始，他们已经从旧的“简单训练风格”迁移到使用
Examplefrom spacy.training import Example
example = Example.from_dict(nlp.make_doc(text), annotations)
nlp.update([example])
您可以在spaCy官方网站上参考此页面。
我认为你仍然尝试使用版本2xx方法，你可以尝试这个例子，它适用于当前版本的spacy：
import spacy
from spacy.training.example import Example
nlp = spacy.load("en_core_web_sm")
# Data pelatihan dalam bentuk batch dari objek Example
train_data = [
(text1, {"entities": [(start1, end1, "LABEL1"), (start2, end2, "LABEL2")]}),
(text2, {"entities": [(start3, end3, "LABEL1"), (start4, end4, "LABEL3")]})
]
# Konversi data pelatihan menjadi batch dari objek Example
examples = []
for text, annotations in train_data:
example = Example.from_dict(nlp.make_doc(text), annotations)
examples.append(example)
# Pembaruan model dengan batch dari objek Example
nlp.update(examples, drop=0.5, losses={})