# Evaluating MT Systems: A Theoretical Framework

arXiv: https://arxiv.org/abs/2202.05806 · HTML: https://mlsanity.com/paper/2202.05806/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 1 of its 15 references that are in the map.
- First posted on arXiv 11 Feb 2022.

## Builds on (most cited first, 1 of 1)

- [COMET: A Neural Framework for MT Evaluation](https://mlsanity.com/paper/2009.09025/) (2020)

## Built on by (most cited first, 1 of 1)

- [Towards Large Language Model driven Reference-less Translation Evaluation for English and Indian Languages](https://mlsanity.com/paper/2404.02512/) (2024)

## Abstract (arXiv)

This paper outlines a theoretical framework using which different automatic metrics can be designed for evaluation of Machine Translation systems. It introduces the concept of em cognitive ease which depends on em adequacy and em lack of fluency. Thus, cognitive ease becomes the main parameter to be measured rather than comprehensibility. The framework allows the components of cognitive ease to be broken up and computed based on different linguistic levels etc. Independence of dimensions and linearly combining them provides for a highly modular approach. The paper places the existing automatic methods in an overall framework, to understand them better and to improve upon them in future. It can also be used to evaluate the newer types of MT systems, such as speech to speech translation and discourse translation.