# Translation Canvas: An Explainable Interface to Pinpoint and Analyze Translation Systems

arXiv: https://arxiv.org/abs/2410.10861 · HTML: https://mlsanity.com/paper/2410.10861/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 17 references that are in the map.
- First posted on arXiv 7 Oct 2024.

## Builds on (most cited first, 9 of 9)

- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [A Call for Clarity in Reporting BLEU Scores](https://mlsanity.com/paper/1804.08771/) (2018)
- [COMET: A Neural Framework for MT Evaluation](https://mlsanity.com/paper/2009.09025/) (2020)
- [How Good Are GPT Models at Machine Translation? A Comprehensive Evaluation](https://mlsanity.com/paper/2302.09210/) (2023)
- [Experts, Errors, and Context: A Large-Scale Study of Human Evaluation for Machine Translation](https://mlsanity.com/paper/2104.14478/) (2021)
- [ExplainaBoard: An Explainable Leaderboard for NLP](https://mlsanity.com/paper/2104.06387/) (2021)
- [INSTRUCTSCORE: Explainable Text Generation Evaluation with Finegrained Feedback](https://mlsanity.com/paper/2305.14282/) (2023)
- [Not All Errors are Equal: Learning Text Generation Metrics using Stratified Error Synthesis](https://mlsanity.com/paper/2210.05035/) (2022)
- [SESCORE2: Learning Text Generation Evaluation via Synthesizing Realistic Mistakes](https://mlsanity.com/paper/2212.09305/) (2022)

## Built on by (most cited first, 1 of 1)

- [Memorization Dynamics in Knowledge Distillation for Language Models](https://mlsanity.com/paper/2601.15394/) (2026)

## Abstract (arXiv)

With the rapid advancement of machine translation research, evaluation toolkits have become essential for benchmarking system progress. Tools like COMET and SacreBLEU offer single quality score assessments that are effective for pairwise system comparisons. However, these tools provide limited insights for fine-grained system-level comparisons and the analysis of instance-level defects. To address these limitations, we introduce Translation Canvas, an explainable interface designed to pinpoint and analyze translation systems' performance: 1) Translation Canvas assists machine translation researchers in comprehending system-level model performance by identifying common errors (their frequency and severity) and analyzing relationships between different systems based on various evaluation metrics. 2) It supports fine-grained analysis by highlighting error spans with explanations and selectively displaying systems' predictions. According to human evaluation, Translation Canvas demonstrates superior performance over COMET and SacreBLEU packages under enjoyability and understandability criteria.