# Towards Opening the Black Box of Neural Machine Translation: Source and Target Interpretations of the Transformer

arXiv: https://arxiv.org/abs/2205.11631 · HTML: https://mlsanity.com/paper/2205.11631/

## Key facts

- Cited by 45 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 37 references that are in the map.
- First posted on arXiv 23 May 2022.

## Builds on (most cited first, 28 of 28)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [fairseq: A Fast, Extensible Toolkit for Sequence Modeling](https://mlsanity.com/paper/1904.01038/) (2019)
- [What Does BERT Look At? An Analysis of BERT's Attention](https://mlsanity.com/paper/1906.04341/) (2019)
- [Attention is not Explanation](https://mlsanity.com/paper/1902.10186/) (2019)
- [Quantifying Attention Flow in Transformers](https://mlsanity.com/paper/2005.00928/) (2020)
- [Transformer Interpretability Beyond Attention Visualization](https://mlsanity.com/paper/2012.09838/) (2020)
- [Beyond English-Centric Multilingual Machine Translation](https://mlsanity.com/paper/2010.11125/) (2020)
- [The FLORES-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translation](https://mlsanity.com/paper/2106.03193/) (2021)
- [Is Attention Interpretable?](https://mlsanity.com/paper/1906.03731/) (2019)
- [Generic Attention-model Explainability for Interpreting Bi-Modal and Encoder-Decoder Transformers](https://mlsanity.com/paper/2103.15679/) (2021)
- [The Bottom-up Evolution of Representations in the Transformer: A Study with Machine Translation and Language Modeling Objectives](https://mlsanity.com/paper/1909.01380/) (2019)
- [On Identifiability in Transformers](https://mlsanity.com/paper/1908.04211/) (2019)
- [Context-Aware Neural Machine Translation Learns Anaphora Resolution](https://mlsanity.com/paper/1805.10163/) (2018)
- [Jointly Learning to Align and Translate with Transformer Models](https://mlsanity.com/paper/1909.02074/) (2019)
- [Analyzing the Source and Target Contributions to Predictions in Neural Machine Translation](https://mlsanity.com/paper/2010.10907/) (2020)
- [Domain Robustness in Neural Machine Translation](https://mlsanity.com/paper/1911.03109/) (2019)
- [Measuring the Mixing of Contextual Information in the Transformer](https://mlsanity.com/paper/2203.04212/) (2022)
- [Incorporating Residual and Normalization Layers into Analysis of Masked Language Models](https://mlsanity.com/paper/2109.07152/) (2021)
- [Accurate Word Alignment Induction from Neural Machine Translation](https://mlsanity.com/paper/2004.14837/) (2020)
- [Adding Interpretable Attention to Neural Translation Models Improves Word Alignment](https://mlsanity.com/paper/1901.11359/) (2019)
- [Saliency-driven Word Alignment Interpretation for Neural Machine Translation](https://mlsanity.com/paper/1906.10282/) (2019)
- [Language Modeling, Lexical Translation, Reordering: The Training Process of NMT through the Lens of Classical SMT](https://mlsanity.com/paper/2109.01396/) (2021)
- [Attention Weights in Transformer NMT Fail Aligning Words Between Sequences but Largely Explain Model Predictions](https://mlsanity.com/paper/2109.05853/) (2021)

## Built on by (most cited first, 30 of 45)

- [Hallucinations in Large Multilingual Translation Models](https://mlsanity.com/paper/2303.16104/) (2023)
- [Inseq: An Interpretability Toolkit for Sequence Generation Models](https://mlsanity.com/paper/2302.13942/) (2023)
- [Detecting and Mitigating Hallucinations in Machine Translation: Model Internal Workings Alone Do Well, Sentence Similarity Even Better](https://mlsanity.com/paper/2212.08597/) (2022)
- [On Early Detection of Hallucinations in Factual Question Answering](https://mlsanity.com/paper/2312.14183/) (2023)
- [Explaining How Transformers Use Context to Build Predictions](https://mlsanity.com/paper/2305.12535/) (2023)
- [HalOmi: A Manually Annotated Benchmark for Multilingual Hallucination and Omission Detection in Machine Translation](https://mlsanity.com/paper/2305.11746/) (2023)
- [Attention as a Guide for Simultaneous Speech Translation](https://mlsanity.com/paper/2212.07850/) (2022)
- [Improving Translation Faithfulness of Large Language Models via Augmenting Instructions](https://mlsanity.com/paper/2308.12674/) (2023)
- [Toxicity in Multilingual Machine Translation at Scale](https://mlsanity.com/paper/2210.03070/) (2022)
- [Multilingual Holistic Bias: Extending Descriptors and Patterns to Unveil Demographic Biases in Languages at Scale](https://mlsanity.com/paper/2305.13198/) (2023)
- [DecoderLens: Layerwise Interpretation of Encoder-Decoder Transformers](https://mlsanity.com/paper/2310.03686/) (2023)
- [A Tale of Pronouns: Interpretability Informs Gender Bias Mitigation for Fairer Instruction-Tuned Machine Translation](https://mlsanity.com/paper/2310.12127/) (2023)
- [An Audit on the Perspectives and Challenges of Hallucinations in NLP](https://mlsanity.com/paper/2404.07461/) (2024)
- [Optimal Transport for Unsupervised Hallucination Detection in Neural Machine Translation](https://mlsanity.com/paper/2212.09631/) (2022)
- [Interpretation Meets Safety: A Survey on Interpretation Methods and Tools for Improving LLM Safety](https://mlsanity.com/paper/2506.05451/) (2025)
- [On Measuring Context Utilization in Document-Level MT Systems](https://mlsanity.com/paper/2402.01404/) (2024)
- [Enhanced Hallucination Detection in Neural Machine Translation through Simple Detector Aggregation](https://mlsanity.com/paper/2402.13331/) (2024)
- [Detecting Hallucination and Coverage Errors in Retrieval Augmented Generation for Controversial Topics](https://mlsanity.com/paper/2403.08904/) (2024)
- [Quantifying the Plausibility of Context Reliance in Neural Machine Translation](https://mlsanity.com/paper/2310.01188/) (2023)
- [Perturbation-based QE: An Explainable, Unsupervised Word-level Quality Estimation Method for Blackbox Machine Translation](https://mlsanity.com/paper/2305.07457/) (2023)
- [Mitigating Hallucinated Translations in Large Language Models with Hallucination-focused Preference Optimization](https://mlsanity.com/paper/2501.17295/) (2025)
- [Hallucinated Span Detection with Multi-View Attention Features](https://mlsanity.com/paper/2504.04335/) (2025)
- [Added Toxicity Mitigation at Inference Time for Multimodal and Massively Multilingual Translation](https://mlsanity.com/paper/2311.06532/) (2023)
- [Two Intermediate Translations Are Better Than One: Fine-tuning LLMs for Document-level Translation Refinement](https://mlsanity.com/paper/2504.05614/) (2025)
- [Isotropy, Clusters, and Classifiers](https://mlsanity.com/paper/2402.03191/) (2024)
- [Analyzing the Attention Heads for Pronoun Disambiguation in Context-aware Machine Translation Models](https://mlsanity.com/paper/2412.11187/) (2024)
- [Jacobian Scopes: token-level causal attributions in LLMs](https://mlsanity.com/paper/2601.16407/) (2026)
- [ReSeTOX: Re-learning attention weights for toxicity mitigation in machine translation](https://mlsanity.com/paper/2305.11761/) (2023)
- [Counterfactuals As a Means for Evaluating Faithfulness of Attribution Methods in Autoregressive Language Models](https://mlsanity.com/paper/2408.11252/) (2024)
- [The Unheard Alternative: Contrastive Explanations for Speech-to-Text Models](https://mlsanity.com/paper/2509.26543/) (2025)

## Abstract (arXiv)

In Neural Machine Translation (NMT), each token prediction is conditioned on the source sentence and the target prefix (what has been previously translated at a decoding step). However, previous work on interpretability in NMT has mainly focused solely on source sentence tokens' attributions. Therefore, we lack a full understanding of the influences of every input token (source sentence and target prefix) in the model predictions. In this work, we propose an interpretability method that tracks input tokens' attributions for both contexts. Our method, which can be extended to any encoder-decoder Transformer-based model, allows us to better comprehend the inner workings of current NMT models. We apply the proposed method to both bilingual and multilingual Transformers and present insights into their behaviour.