# Translation Error Detection as Rationale Extraction

arXiv: https://arxiv.org/abs/2108.12197 · HTML: https://mlsanity.com/paper/2108.12197/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 40 references that are in the map.
- First posted on arXiv 27 Aug 2021.

## Builds on (most cited first, 29 of 29)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://mlsanity.com/paper/1312.6034/) (2013)
- [Attention is not Explanation](https://mlsanity.com/paper/1902.10186/) (2019)
- [Deep Learning and the Information Bottleneck Principle](https://mlsanity.com/paper/1503.02406/) (2015)
- [The Mythos of Model Interpretability](https://mlsanity.com/paper/1606.03490/) (2016)
- [Attention is not not Explanation](https://mlsanity.com/paper/1908.04626/) (2019)
- [Towards Faithfully Interpretable NLP Systems: How should we define and evaluate faithfulness?](https://mlsanity.com/paper/2004.03685/) (2020)
- [ERASER: A Benchmark to Evaluate Rationalized NLP Models](https://mlsanity.com/paper/1911.03429/) (2019)
- [Is Attention Interpretable?](https://mlsanity.com/paper/1906.03731/) (2019)
- [Experts, Errors, and Context: A Large-Scale Study of Human Evaluation for Machine Translation](https://mlsanity.com/paper/2104.14478/) (2021)
- [Rationalizing Neural Predictions](https://mlsanity.com/paper/1606.04155/) (2016)
- [Unsupervised Quality Estimation for Neural Machine Translation](https://mlsanity.com/paper/2005.10608/) (2020)
- [A Diagnostic Study of Explainability Techniques for Text Classification](https://mlsanity.com/paper/2009.13295/) (2020)
- [The Bottom-up Evolution of Representations in the Transformer: A Study with Machine Translation and Language Modeling Objectives](https://mlsanity.com/paper/1909.01380/) (2019)
- [Learning to Faithfully Rationalize by Construction](https://mlsanity.com/paper/2005.00115/) (2020)
- [Restricting the Flow: Information Bottlenecks for Attribution](https://mlsanity.com/paper/2001.00396/) (2020)
- [Transformer visualization via dictionary learning: contextualized embedding as a linear superposition of transformer factors](https://mlsanity.com/paper/2103.15949/) (2021)
- [Rethinking Cooperative Rationalization: Introspective Extraction and Complement Control](https://mlsanity.com/paper/1910.13294/) (2019)
- [Paragraph-level Rationale Extraction through Regularization: A case study on European Court of Human Rights Cases](https://mlsanity.com/paper/2103.13084/) (2021)
- [How do Decisions Emerge across Layers in Neural Models? Interpretation with Differentiable Masking](https://mlsanity.com/paper/2004.14992/) (2020)
- [TransQuest: Translation Quality Estimation with Cross-lingual Transformers](https://mlsanity.com/paper/2011.01536/) (2020)
- [MLQE-PE: A Multilingual Quality Estimation and Post-Editing Dataset](https://mlsanity.com/paper/2010.04480/) (2020)
- [TransQuest at WMT2020: Sentence-Level Direct Assessment](https://mlsanity.com/paper/2010.05318/) (2020)
- [Quality Estimation without Human-labeled Data](https://mlsanity.com/paper/2102.04020/) (2021)
- [Confidence through Attention](https://mlsanity.com/paper/1710.03743/) (2017)
- [Zero-shot Sequence Labeling for Transformer-based Sentence Classifiers](https://mlsanity.com/paper/2103.14465/) (2021)

## Built on by (most cited first, 22 of 22)

- [The Eval4NLP Shared Task on Explainable Quality Estimation: Overview and Results](https://mlsanity.com/paper/2110.04392/) (2021)
- [Global Explainability of BERT-Based Evaluation Metrics by Disentangling along Linguistic Factors](https://mlsanity.com/paper/2110.04399/) (2021)
- [Rationalization for Explainable NLP: A Survey](https://mlsanity.com/paper/2301.08912/) (2023)
- [Learning to Scaffold: Optimizing Model Explanations for Teaching](https://mlsanity.com/paper/2204.10810/) (2022)
- [Towards Explainable Evaluation Metrics for Natural Language Generation](https://mlsanity.com/paper/2203.11131/) (2022)
- [The Inside Story: Towards Better Understanding of Machine Translation Neural Evaluation Metrics](https://mlsanity.com/paper/2305.11806/) (2023)
- [Towards Explainable Evaluation Metrics for Machine Translation](https://mlsanity.com/paper/2306.13041/) (2023)
- [Incorporating Attribution Importance for Improving Faithfulness Metrics](https://mlsanity.com/paper/2305.10496/) (2023)
- [Explainability in Practice: A Survey of Explainable NLP Across Various Domains](https://mlsanity.com/paper/2502.00837/) (2025)
- [Beyond Correlation: Interpretable Evaluation of Machine Translation Metrics](https://mlsanity.com/paper/2410.05183/) (2024)
- [RAMP: Retrieval and Attribute-Marking Enhanced Prompting for Attribute-Controlled Translation](https://mlsanity.com/paper/2305.17131/) (2023)
- [AskQE: Question Answering as Automatic Evaluation for Machine Translation](https://mlsanity.com/paper/2504.11582/) (2025)
- [A Multilingual Perspective Towards the Evaluation of Attribution Methods in Natural Language Inference](https://mlsanity.com/paper/2204.05428/) (2022)
- [Towards Unsupervised Recognition of Token-level Semantic Differences in Related Documents](https://mlsanity.com/paper/2305.13303/) (2023)
- [Should I Share this Translation? Evaluating Quality Feedback for User Reliance on Machine Translation](https://mlsanity.com/paper/2505.24683/) (2025)
- [Word-Level ASR Quality Estimation for Efficient Corpus Sampling and Post-Editing through Analyzing Attentions of a Reference-Free Metric](https://mlsanity.com/paper/2401.11268/) (2024)
- [SOLD: Sinhala Offensive Language Dataset](https://mlsanity.com/paper/2212.00851/) (2022)
- [Does Summary Evaluation Survive Translation to Other Languages?](https://mlsanity.com/paper/2109.08129/) (2021)
- [Consistency and Coherence from Points of Contextual Similarity](https://mlsanity.com/paper/2112.11638/) (2021)
- [Explanation Regularisation through the Lens of Attributions](https://mlsanity.com/paper/2407.16693/) (2024)
- [Pingala: Prosody-Aware Decoding for Sanskrit Poetry Generation](https://mlsanity.com/paper/2603.24413/) (2026)
- [Evaluating Explainable AI Attribution Methods in Neural Machine Translation via Attention-Guided Knowledge Distillation](https://mlsanity.com/paper/2603.11342/) (2026)

## Abstract (arXiv)

Recent Quality Estimation (QE) models based on multilingual pre-trained representations have achieved very competitive results when predicting the overall quality of translated sentences. Predicting translation errors, i.e. detecting specifically which words are incorrect, is a more challenging task, especially with limited amounts of training data. We hypothesize that, not unlike humans, successful QE models rely on translation errors to predict overall sentence quality. By exploring a set of feature attribution methods that assign relevance scores to the inputs to explain model predictions, we study the behaviour of state-of-the-art sentence-level QE models and show that explanations (i.e. rationales) extracted from these models can indeed be used to detect translation errors. We therefore (i) introduce a novel semi-supervised method for word-level QE and (ii) propose to use the QE task as a new benchmark for evaluating the plausibility of feature attribution, i.e. how interpretable model explanations are to humans.