Translation Memory Retrieval Met

Translation Memory Similarity Metrics

所有的评价指标都获取两个输入（M与C），M是待翻译的句子，C是来自TM库中的源端候选

Percent Match

percent match.png

Weighted Percent Match

上面方法的缺点在于对所有的unigram都同等的看待，但对于译者来说，并不是所有词对他所起的帮助都是等同的，那些不确定如何翻译的部分才是最有价值的
Weighted percent match (WPM) uses inverse document frequency(IDF) as a proxy for trying to weight words based on how much value their translations are expected to provide to translators.
那些出现频率高的词往往容易翻，而那些出现频率低的词更难翻好，因此其价值也就越大

image.png

Edit Distance

上面两种方法的缺点在于都只考虑了词的覆盖度，而没有考虑词的上下文，而相同的词在不同的上下文的语境下翻译出来的结果会有很大的差别。
Thus, a TM metric that matches sentences on more than just (weighted) percentage coverage of lexical items can be expected to perform better for TM bank evaluation and retrieval.

image.png

N-Gram Precision

Although ED takes context into account, it does not emphasize local context in matching certainhigh-value words and phrases as much as metrics that capture n-gram precision between the MTBT workload sentence and candidate source-side sentences from the TMB.
有点类似BLEU值的评测
Perhaps the most important is that TM fuzzy matching has to be able to operate at a sentence-to-sentence level where as automated MT evaluation metrics such as BLEU score are intended to operate over a whole corpus.

image.png

选择算术平均会比几何平均更好，更加贴合当前的任务。
对Z值的设定是一个召回率与准确率的trade off，Z值设成1会倾向于得到更长的句子（考虑召回率），而设成0会倾向于得到更短的句子（考虑精确率），作者最后实现设置的值为0.75

Weighted N-Gram Precision

与提高PM到WPM的考虑相似，对NGP也做相同的处理

image.png

Modified Weighted N-Gram Precision

对n-gram也根据这个n做一个加权
Note that in Equation 6 each wpn contributes equally to the average. Modified Weighted NGram Precision (MWNGP) improves on WNGP by weighting the contribution of each wpn so that shorter n-grams contribute more than longer ngrams. The intuition is that for TM settings, getting more high-value shorter n-gram matches atthe expense of fewer longer n-gram matches willbe more helpful since translators will get relatively more assistance from seeing new high-value vocabulary.

image.png