# Regularizing Neural Machine Translation by Target-bidirectional Agreement

arXiv: https://arxiv.org/abs/1808.04064 · HTML: https://mlsanity.com/paper/1808.04064/

## Key facts

- Cited by 40 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Aug 2018.

## Built on by (most cited first, 30 of 40)

- [Word Alignment by Fine-tuning Embeddings on Parallel Corpora](https://mlsanity.com/paper/2101.08231/) (2021)
- [Domain Adaptation and Multi-Domain Adaptation for Neural Machine Translation: A Survey](https://mlsanity.com/paper/2104.06951/) (2021)
- [Handwritten Mathematical Expression Recognition with Bidirectionally Trained Transformer](https://mlsanity.com/paper/2105.02412/) (2021)
- [Meet in the Middle: A New Pre-training Paradigm](https://mlsanity.com/paper/2303.07295/) (2023)
- [Arrows of Time for Large Language Models](https://mlsanity.com/paper/2401.17505/) (2024)
- [Regularizing End-to-End Speech Translation with Triangular Decomposition Agreement](https://mlsanity.com/paper/2112.10991/) (2021)
- [Transductive Learning for Unsupervised Text Style Transfer](https://mlsanity.com/paper/2109.07812/) (2021)
- [The Belief State Transformer](https://mlsanity.com/paper/2410.23506/) (2024)
- [Toward Learning Robust and Invariant Representations with Alignment Regularization and Data Augmentation](https://mlsanity.com/paper/2206.01909/) (2022)
- [Can Diffusion Model Achieve Better Performance in Text Generation? Bridging the Gap between Training and Inference!](https://mlsanity.com/paper/2305.04465/) (2023)
- [Generic resources are what you need: Style transfer tasks without task-specific parallel training data](https://mlsanity.com/paper/2109.04543/) (2021)
- [Back-Training excels Self-Training at Unsupervised Domain Adaptation of Question Generation and Passage Retrieval](https://mlsanity.com/paper/2104.08801/) (2021)
- [Guiding Teacher Forcing with Seer Forcing for Neural Machine Translation](https://mlsanity.com/paper/2106.06751/) (2021)
- [Rethinking Data Augmentation for Low-Resource Neural Machine Translation: A Multi-Task Learning Approach](https://mlsanity.com/paper/2109.03645/) (2021)
- [NVIDIA NeMo Neural Machine Translation Systems for English-German and English-Russian News and Biomedical Tasks at WMT21](https://mlsanity.com/paper/2111.08634/) (2021)
- [NAST: A Non-Autoregressive Generator with Word Alignment for Unsupervised Text Style Transfer](https://mlsanity.com/paper/2106.02210/) (2021)
- [Image Captioning via Compact Bidirectional Architecture](https://mlsanity.com/paper/2201.01984/) (2022)
- [Confidence Based Bidirectional Global Context Aware Training Framework for Neural Machine Translation](https://mlsanity.com/paper/2202.13663/) (2022)
- [Self-Supervised Knowledge Assimilation for Expert-Layman Text Style Transfer](https://mlsanity.com/paper/2110.02950/) (2021)
- [Audience-Centric Natural Language Generation via Style Infusion](https://mlsanity.com/paper/2301.10283/) (2023)
- [Federated Nearest Neighbor Machine Translation](https://mlsanity.com/paper/2302.12211/) (2023)
- [Modeling Coverage for Non-Autoregressive Neural Machine Translation](https://mlsanity.com/paper/2104.11897/) (2021)
- [Preventing Author Profiling through Zero-Shot Multilingual Back-Translation](https://mlsanity.com/paper/2109.09133/) (2021)
- [On Synthetic Data for Back Translation](https://mlsanity.com/paper/2310.13675/) (2023)
- [Syntax Matters! Syntax-Controlled in Text Style Transfer](https://mlsanity.com/paper/2108.05869/) (2021)
- [3M: Multi-style image caption generation using Multi-modality features under Multi-UPDOWN model](https://mlsanity.com/paper/2103.11186/) (2021)
- [Unsupervised Sentence Simplification via Dependency Parsing](https://mlsanity.com/paper/2206.12261/) (2022)
- [Bidirectional Awareness Induction in Autoregressive Seq2Seq Models](https://mlsanity.com/paper/2408.13959/) (2024)
- [SE-DAE: Style-Enhanced Denoising Auto-Encoder for Unsupervised Text Style Transfer](https://mlsanity.com/paper/2104.12977/) (2021)
- [Probability Consistency in Large Language Models: Theoretical Foundations Meet Empirical Discrepancies](https://mlsanity.com/paper/2505.08739/) (2025)

## Abstract (arXiv)

Although Neural Machine Translation (NMT) has achieved remarkable progress in the past several years, most NMT systems still suffer from a fundamental shortcoming as in other sequence generation tasks: errors made early in generation process are fed as inputs to the model and can be quickly amplified, harming subsequent sequence generation. To address this issue, we propose a novel model regularization method for NMT training, which aims to improve the agreement between translations generated by left-to-right (L2R) and right-to-left (R2L) NMT decoders. This goal is achieved by introducing two Kullback-Leibler divergence regularization terms into the NMT training objective to reduce the mismatch between output probabilities of L2R and R2L models. In addition, we also employ a joint training strategy to allow L2R and R2L models to improve each other in an interactive update process. Experimental results show that our proposed method significantly outperforms state-of-the-art baselines on Chinese-English and English-German translation tasks.