NLTK :: nltk.translate.meteor

2023-08-11 15:23| 来源: 网络整理| 查看: 265

Calculates METEOR score for hypothesis with multiple references as described in “Meteor: An Automatic Metric for MT Evaluation with HighLevels of Correlation with Human Judgments” by Alon Lavie and Abhaya Agarwal, in Proceedings of ACL. https://www.cs.cmu.edu/~alavie/METEOR/pdf/Lavie-Agarwal-2007-METEOR.pdf

In case of multiple references the best score is chosen. This method iterates over single_meteor_score and picks the best pair among all the references for a given hypothesis

>>> hypothesis1 = ['It', 'is', 'a', 'guide', 'to', 'action', 'which', 'ensures', 'that', 'the', 'military', 'always', 'obeys', 'the', 'commands', 'of', 'the', 'party'] >>> hypothesis2 = ['It', 'is', 'to', 'insure', 'the', 'troops', 'forever', 'hearing', 'the', 'activity', 'guidebook', 'that', 'party', 'direct'] >>> reference1 = ['It', 'is', 'a', 'guide', 'to', 'action', 'that', 'ensures', 'that', 'the', 'military', 'will', 'forever', 'heed', 'Party', 'commands'] >>> reference2 = ['It', 'is', 'the', 'guiding', 'principle', 'which', 'guarantees', 'the', 'military', 'forces', 'always', 'being', 'under', 'the', 'command', 'of', 'the', 'Party'] >>> reference3 = ['It', 'is', 'the', 'practical', 'guide', 'for', 'the', 'army', 'always', 'to', 'heed', 'the', 'directions', 'of', 'the', 'party'] >>> round(meteor_score([reference1, reference2, reference3], hypothesis1),4) 0.6944

If there is no words match during the alignment the method returns the score as 0. We can safely return a zero instead of raising a division by zero error as no match usually implies a bad translation.

>>> round(meteor_score([['this', 'is', 'a', 'cat']], ['non', 'matching', 'hypothesis']),4) 0.0 Parameters

references (Iterable[Iterable[str]]) – pre-tokenized reference sentences

hypothesis (Iterable[str]) – a pre-tokenized hypothesis sentence

preprocess (Callable[[str], str]) – preprocessing function (default str.lower)

stemmer (StemmerI) – nltk.stem.api.StemmerI object (default PorterStemmer())

wordnet (WordNetCorpusReader) – a wordnet corpus reader object (default nltk.corpus.wordnet)

alpha (float) – parameter for controlling relative weights of precision and recall.

beta (float) – parameter for controlling shape of penalty as a function of as a function of fragmentation.

gamma (float) – relative weight assigned to fragmentation penalty.

Returns

The sentence-level METEOR score.

Return type

float

【本文地址】

公司简介

联系我们