# Encouraging Neural Machine Translation to Satisfy Terminology Constraints

arXiv: https://arxiv.org/abs/2106.03730 · HTML: https://mlsanity.com/paper/2106.03730/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 19 references that are in the map.
- First posted on arXiv 7 Jun 2021.

## Builds on (most cited first, 14 of 14)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [Effective Approaches to Attention-based Neural Machine Translation](https://mlsanity.com/paper/1508.04025/) (2015)
- [Lexically Constrained Decoding for Sequence Generation Using Grid Beam Search](https://mlsanity.com/paper/1704.07138/) (2017)
- [Levenshtein Transformer](https://mlsanity.com/paper/1905.11006/) (2019)
- [Fast Lexically Constrained Decoding with Dynamic Beam Allocation for Neural Machine Translation](https://mlsanity.com/paper/1804.06609/) (2018)
- [Training Neural Machine Translation To Apply Terminology Constraints](https://mlsanity.com/paper/1906.01105/) (2019)
- [Code-Switching for Enhancing NMT with Pre-Specified Translation](https://mlsanity.com/paper/1904.09107/) (2019)
- [Lexically Constrained Neural Machine Translation with Levenshtein Transformer](https://mlsanity.com/paper/2004.12681/) (2020)
- [Neural Machine Translation Decoding with Terminology Constraints](https://mlsanity.com/paper/1805.03750/) (2018)
- [SYSTRAN's Pure Neural Machine Translation Systems](https://mlsanity.com/paper/1610.05540/) (2016)
- [Towards one-shot learning for rare-word translation with external experts](https://mlsanity.com/paper/1809.03182/) (2018)

## Built on by (most cited first, 10 of 10)

- [Mitigating Large Language Model Hallucination with Faithful Finetuning](https://mlsanity.com/paper/2406.11267/) (2024)
- [Toward Unified Controllable Text Generation via Regular Expression Instruction](https://mlsanity.com/paper/2309.10447/) (2023)
- [Efficient Terminology Integration for LLM-based Translation in Specialized Domains](https://mlsanity.com/paper/2410.15690/) (2024)
- [Neighbors Are Not Strangers: Improving Non-Autoregressive Translation under Low-Frequency Lexical Constraints](https://mlsanity.com/paper/2204.13355/) (2022)
- [Controlling Extra-Textual Attributes about Dialogue Participants -- A Case Study of English-to-Polish Neural Machine Translation](https://mlsanity.com/paper/2205.04747/) (2022)
- [Optimizing Rare Word Accuracy in Direct Speech Translation with a Retrieval-and-Demonstration Approach](https://mlsanity.com/paper/2409.09009/) (2024)
- [Locate-and-Focus: Enhancing Terminology Translation in Speech Language Models](https://mlsanity.com/paper/2507.18263/) (2025)
- [Lingua Custodia's participation at the WMT 2021 Machine Translation using Terminologies shared task](https://mlsanity.com/paper/2111.02120/) (2021)
- [Lingua Custodi's participation at the WMT 2025 Terminology shared task](https://mlsanity.com/paper/2510.17504/) (2025)
- [Worlds Within Words: Translating Culture in Ancient Chinese Texts with Multi-Agent Coordination](https://mlsanity.com/paper/2606.01276/) (2026)

## Abstract (arXiv)

We present a new approach to encourage neural machine translation to satisfy lexical constraints. Our method acts at the training step and thereby avoiding the introduction of any extra computational overhead at inference step. The proposed method combines three main ingredients. The first one consists in augmenting the training data to specify the constraints. Intuitively, this encourages the model to learn a copy behavior when it encounters constraint terms. Compared to previous work, we use a simplified augmentation strategy without source factors. The second ingredient is constraint token masking, which makes it even easier for the model to learn the copy behavior and generalize better. The third one, is a modification of the standard cross entropy loss to bias the model towards assigning high probabilities to constraint words. Empirical results show that our method improves upon related baselines in terms of both BLEU score and the percentage of generated constraint terms.