# Synchronous Bidirectional Neural Machine Translation

arXiv: https://arxiv.org/abs/1905.04847 · HTML: https://mlsanity.com/paper/1905.04847/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 May 2019.

## Built on by (most cited first, 27 of 27)

- [Handwritten Mathematical Expression Recognition with Bidirectionally Trained Transformer](https://mlsanity.com/paper/2105.02412/) (2021)
- [Meet in the Middle: A New Pre-training Paradigm](https://mlsanity.com/paper/2303.07295/) (2023)
- [Reducing Position Bias in Simultaneous Machine Translation with Length-Aware Framework](https://mlsanity.com/paper/2203.09053/) (2022)
- [Discovering Non-monotonic Autoregressive Orderings with Variational Inference](https://mlsanity.com/paper/2110.15797/) (2021)
- [Other Roles Matter! Enhancing Role-Oriented Dialogue Summarization via Role Interactions](https://mlsanity.com/paper/2205.13190/) (2022)
- [Capturing Event Argument Interaction via A Bi-Directional Entity-Level Recurrent Decoder](https://mlsanity.com/paper/2107.00189/) (2021)
- [Scheduled Sampling Based on Decoding Steps for Neural Machine Translation](https://mlsanity.com/paper/2108.12963/) (2021)
- [Confidence-Aware Scheduled Sampling for Neural Machine Translation](https://mlsanity.com/paper/2107.10427/) (2021)
- [Guiding Teacher Forcing with Seer Forcing for Neural Machine Translation](https://mlsanity.com/paper/2106.06751/) (2021)
- [Image Captioning via Compact Bidirectional Architecture](https://mlsanity.com/paper/2201.01984/) (2022)
- [Confidence Based Bidirectional Global Context Aware Training Framework for Neural Machine Translation](https://mlsanity.com/paper/2202.13663/) (2022)
- [Joint Generation of Captions and Subtitles with Dual Decoding](https://mlsanity.com/paper/2205.06522/) (2022)
- [Efficient Modeling of Future Context for Image Captioning](https://mlsanity.com/paper/2207.10897/) (2022)
- [Twist Decoding: Diverse Generators Guide Each Other](https://mlsanity.com/paper/2205.09273/) (2022)
- [Accurate Knowledge Distillation with n-best Reranking](https://mlsanity.com/paper/2305.12057/) (2023)
- [Neural Machine Translation with Dynamic Graph Convolutional Decoder](https://mlsanity.com/paper/2305.17698/) (2023)
- [One Source, Two Targets: Challenges and Rewards of Dual Decoding](https://mlsanity.com/paper/2109.10197/) (2021)
- [Unifying the Convergences in Multilingual Neural Machine Translation](https://mlsanity.com/paper/2205.01620/) (2022)
- [Bidirectional Awareness Induction in Autoregressive Seq2Seq Models](https://mlsanity.com/paper/2408.13959/) (2024)
- [Reverse Modeling in Large Language Models](https://mlsanity.com/paper/2410.09817/) (2024)
- [Enhancing the Transformer Decoder with Transition-based Syntax](https://mlsanity.com/paper/2101.12640/) (2021)
- [Spiral Language Modeling](https://mlsanity.com/paper/2112.10543/) (2021)
- [Refining the state-of-the-art in Machine Translation, optimizing NMT for the JA <-> EN language pair by leveraging personal domain expertise](https://mlsanity.com/paper/2202.11669/) (2022)
- [Look Backward and Forward: Self-Knowledge Distillation with Bidirectional Decoder for Neural Machine Translation](https://mlsanity.com/paper/2203.05248/) (2022)
- [A Framework for Bidirectional Decoding: Case Study in Morphological Inflection](https://mlsanity.com/paper/2305.12580/) (2023)
- [Guiding AMR Parsing with Reverse Graph Linearization](https://mlsanity.com/paper/2310.08860/) (2023)
- [Leveraging Diverse Modeling Contexts with Collaborating Learning for Neural Machine Translation](https://mlsanity.com/paper/2402.18428/) (2024)

## Abstract (arXiv)

Existing approaches to neural machine translation (NMT) generate the target language sequence token by token from left to right. However, this kind of unidirectional decoding framework cannot make full use of the target-side future contexts which can be produced in a right-to-left decoding direction, and thus suffers from the issue of unbalanced outputs. In this paper, we introduce a synchronous bidirectional neural machine translation (SB-NMT) that predicts its outputs using left-to-right and right-to-left decoding simultaneously and interactively, in order to leverage both of the history and future information at the same time. Specifically, we first propose a new algorithm that enables synchronous bidirectional decoding in a single model. Then, we present an interactive decoding model in which left-to-right (right-to-left) generation does not only depend on its previously generated outputs, but also relies on future contexts predicted by right-to-left (left-to-right) decoding. We extensively evaluate the proposed SB-NMT model on large-scale NIST Chinese-English, WMT14 English-German, and WMT18 Russian-English translation tasks. Experimental results demonstrate that our model achieves significant improvements over the strong Transformer model by 3.92, 1.49 and 1.04 BLEU points respectively, and obtains the state-of-the-art performance on Chinese-English and English-German translation tasks.