类型错误：“假设”需要预先标记化的假设（Iterable[str]）：

Question

我正在尝试计算以下内容的 Meteor 分数：

print (nltk.translate.meteor_score.meteor_score(
    ["this is an apple", "that is an apple"], "an apple on this tree"))

但是我每次都会收到此错误，并且不知道如何修复它。

TypeError: "hypothesis" expects pre-tokenized hypothesis (Iterable[str]): an apple on this tree

我还尝试将“这棵树上的苹果”放入列表中

    from nltk.translate.meteor_score import meteor_score
import nltk 
print (nltk.translate.meteor_score.meteor_score(
    ["this is an apple", "that is an apple"], ["an apple on this tree"]))

但是它给了我这个错误。

TypeError: "reference" expects pre-tokenized reference (Iterable[str]): this is an apple

Answer 1

实际上，我认为问题的正确答案是在调用函数之前对句子进行标记。例如：

    for line in zip(refs, hypos):
        ref = word_tokenize(line[0])
        hypo = word_tokenize(line[1])
        m_score += meteor_score([ref], hypo)

其中ref和hypo是句子字符串。

Answer 2

查看库代码，看起来假设应该是一个可迭代的。 https://www.nltk.org/_modules/nltk/translate/meteor_score.html。错误来自：

if isinstance(hypothesis, str):
        raise TypeError(
            f'"hypothesis" expects pre-tokenized hypothesis (Iterable[str]): {hypothesis}'
        )

尝试将“这棵树上的苹果”放入列表中。

Answer 3

遇到同样的错误可以用官方文档修复

from nltk.tokenize import word_tokenize
nltk.download('punkt')
nltk.download('wordnet')
hypos = "The quick brown fox jumps over the lazy dog" 
refs = "A fast brown fox leaps over a lazy dog" 

score = meteor_score([word_tokenize(refs)], word_tokenize(hypos)) 

print(f"METEOR Score: {score:.4f}")