我如何在PyCharm中运行此Polyglot令牌/标签提取器？

Question

我正在评估各种命名实体识别（NER）库，并且正在尝试Polyglot。

一切似乎都进展顺利，但说明告诉我在命令提示符下使用此行：

!polyglot --lang en tokenize --input testdata/cricket.txt |  polyglot --lang en ner | tail -n 20

...这应该在示例中提供此输出：

,               O
which           O
was             O
equalled        O
five            O
days            O
ago             O
by              O
South           I-LOC
Africa          I-LOC
in              O
their           O
victory         O
over            O
West            I-ORG
Indies          I-ORG
in              O
Sydney          I-LOC
.               O

这正是我的项目所需的输出，它的工作方式与我需要的工作完全相同；但是，我需要在我的PyCharm界面而不是命令行中运行它，并将结果存储在pandas数据框中。如何翻译该命令？

Answer 1

假设polyglot已正确安装，并且在pycharm中选择了正确的环境。如果没有，请在有必要要求的new conda environment中安装polyglot。创建一个新项目，然后在pycharm中选择该现有的conda环境。如果language embeddings，ner型号不是downloaded，则应下载它们。

代码：

from polyglot.text import Text

blob = """, which was equalled five days ago by South Africa in the victory over West Indies in Sydney."""
text = Text(blob)
text.language = "en"


## As list all detected entities
print("As list all detected entities")
print(text.entities)

print()

## Separately shown detected entities
print("Separately shown detected entities")
for entity in text.entities:
    print(entity.tag, entity)

print()

## Tokenized words of sentence
print("Tokenized words of sentence")
print(text.words)

print()

## For each token try named entity recognition.
## Not very reliable it detects some words as not English and tries other languages.
## If other embeddings are not installed or text.language = "en" is commented then it may give error.
print("For each token try named entity recognition")
for word in text.words:
    text = Text(word)
    text.language = "en"

    ## Separately
    for entity in text.entities:
        print(entity.tag, entity)

输出：

As list all detected entities
[I-LOC(['South', 'Africa']), I-ORG(['West', 'Indies']), I-LOC(['Sydney'])]

Separately shown detected entities
I-LOC ['South', 'Africa']
I-ORG ['West', 'Indies']
I-LOC ['Sydney']

Tokenized words of sentence
[',', 'which', 'was', 'equalled', 'five', 'days', 'ago', 'by', 'South', 'Africa', 'in', 'the', 'victory', 'over', 'West', 'Indies', 'in', 'Sydney', '.']

For each token try named entity recognition
I-LOC ['Africa']
I-PER ['Sydney']