# Machine Translation Evaluation using Bi-directional Entailment

arXiv: https://arxiv.org/abs/1911.00681 · HTML: https://mlsanity.com/paper/1911.00681/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Nov 2019.

## Built on by (most cited first, 4 of 4)

- [Not All Errors are Equal: Learning Text Generation Metrics using Stratified Error Synthesis](https://mlsanity.com/paper/2210.05035/) (2022)
- [Multi-turn Dialogue Reading Comprehension with Pivot Turns and Knowledge](https://mlsanity.com/paper/2102.05474/) (2021)
- [LLM-Based Insight Extraction for Contact Center Analytics and Cost-Efficient Deployment](https://mlsanity.com/paper/2503.19090/) (2025)
- [Is Textual Similarity Invariant under Machine Translation? Evidence Based on the Political Manifesto Corpus](https://mlsanity.com/paper/2605.00618/) (2026)

## Abstract (arXiv)

In this paper, we propose a new metric for Machine Translation (MT) evaluation, based on bi-directional entailment. We show that machine generated translation can be evaluated by determining paraphrasing with a reference translation provided by a human translator. We hypothesize, and show through experiments, that paraphrasing can be detected by evaluating entailment relationship in the forward and backward direction. Unlike conventional metrics, like BLEU or METEOR, our approach uses deep learning to determine the semantic similarity between candidate and reference translation for generating scores rather than relying upon simple n-gram overlap. We use BERT's pre-trained implementation of transformer networks, fine-tuned on MNLI corpus, for natural language inferencing. We apply our evaluation metric on WMT'14 and WMT'17 dataset to evaluate systems participating in the translation task and find that our metric has a better correlation with the human annotated score compared to the other traditional metrics at system level.