# Revisiting Negation in Neural Machine Translation

arXiv: https://arxiv.org/abs/2107.12203 · HTML: https://mlsanity.com/paper/2107.12203/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 41 references that are in the map.
- First posted on arXiv 26 Jul 2021.

## Builds on (most cited first, 21 of 21)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [Deep contextualized word representations](https://mlsanity.com/paper/1802.05365/) (2018)
- [A Call for Clarity in Reporting BLEU Scores](https://mlsanity.com/paper/1804.08771/) (2018)
- [Quantifying Attention Flow in Transformers](https://mlsanity.com/paper/2005.00928/) (2020)
- [Fast Lexically Constrained Decoding with Dynamic Beam Allocation for Neural Machine Translation](https://mlsanity.com/paper/1804.06609/) (2018)
- [Modeling Coverage for Neural Machine Translation](https://mlsanity.com/paper/1601.04811/) (2016)
- [On Using Very Large Target Vocabulary for Neural Machine Translation](https://mlsanity.com/paper/1412.2007/) (2014)
- [Pointing the Unknown Words](https://mlsanity.com/paper/1603.08148/) (2016)
- [How Grammatical is Character-level Neural Machine Translation? Assessing MT Quality with Contrastive Translation Pairs](https://mlsanity.com/paper/1612.04629/) (2016)
- [NegBERT: A Transfer Learning Approach for Negation Detection and Scope Resolution](https://mlsanity.com/paper/1911.04211/) (2019)
- [Why Self-Attention? A Targeted Evaluation of Neural Machine Translation Architectures](https://mlsanity.com/paper/1808.08946/) (2018)
- [Neural versus Phrase-Based Machine Translation Quality: a Case Study](https://mlsanity.com/paper/1608.04631/) (2016)
- [It's not a Non-Issue: Negation as a Source of Error in Machine Translation](https://mlsanity.com/paper/2010.05432/) (2020)
- [Sockeye: A Toolkit for Neural Machine Translation](https://mlsanity.com/paper/1712.05690/) (2017)
- [Context Gates for Neural Machine Translation](https://mlsanity.com/paper/1608.06043/) (2016)
- [Confidence through Attention](https://mlsanity.com/paper/1710.03743/) (2017)
- [On the Importance of Word Boundaries in Character-level Neural Machine Translation](https://mlsanity.com/paper/1910.06753/) (2019)
- [Use of Modality and Negation in Semantically-Informed Syntactic MT](https://mlsanity.com/paper/1502.01682/) (2015)
- [Neural Networks for Cross-lingual Negation Scope Detection](https://mlsanity.com/paper/1810.02156/) (2018)

## Built on by (most cited first, 6 of 6)

- [Not another Negation Benchmark: The NaN-NLI Test Suite for Sub-clausal Negation](https://mlsanity.com/paper/2210.03256/) (2022)
- [MENLI: Robust Evaluation Metrics from Natural Language Inference](https://mlsanity.com/paper/2208.07316/) (2022)
- [This is not a Dataset: A Large Negation Benchmark to Challenge Large Language Models](https://mlsanity.com/paper/2310.15941/) (2023)
- [On the Limits of Minimal Pairs in Contrastive Evaluation](https://mlsanity.com/paper/2109.07465/) (2021)
- [Applications and Challenges of Sentiment Analysis in Real-life Scenarios](https://mlsanity.com/paper/2301.09912/) (2023)
- [Thunder-KoNUBench: A Corpus-Aligned Benchmark for Korean Negation Understanding](https://mlsanity.com/paper/2601.04693/) (2026)

## Abstract (arXiv)

In this paper, we evaluate the translation of negation both automatically and manually, in English--German (EN--DE) and English--Chinese (EN--ZH). We show that the ability of neural machine translation (NMT) models to translate negation has improved with deeper and more advanced networks, although the performance varies between language pairs and translation directions. The accuracy of manual evaluation in EN-DE, DE-EN, EN-ZH, and ZH-EN is 95.7%, 94.8%, 93.4%, and 91.7%, respectively. In addition, we show that under-translation is the most significant error type in NMT, which contrasts with the more diverse error profile previously observed for statistical machine translation. To better understand the root of the under-translation of negation, we study the model's information flow and training data. While our information flow analysis does not reveal any deficiencies that could be used to detect or fix the under-translation of negation, we find that negation is often rephrased during training, which could make it more difficult for the model to learn a reliable link between source and target negation. We finally conduct intrinsic analysis and extrinsic probing tasks on negation, showing that NMT models can distinguish negation and non-negation tokens very well and encode a lot of information about negation in hidden states but nevertheless leave room for improvement.