# Sequence Generation: From Both Sides to the Middle

arXiv: https://arxiv.org/abs/1906.09601 · HTML: https://mlsanity.com/paper/1906.09601/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Jun 2019.

## Built on by (most cited first, 5 of 5)

- [Guiding Teacher Forcing with Seer Forcing for Neural Machine Translation](https://mlsanity.com/paper/2106.06751/) (2021)
- [Bidirectional Awareness Induction in Autoregressive Seq2Seq Models](https://mlsanity.com/paper/2408.13959/) (2024)
- [Spiral Language Modeling](https://mlsanity.com/paper/2112.10543/) (2021)
- [A Framework for Bidirectional Decoding: Case Study in Morphological Inflection](https://mlsanity.com/paper/2305.12580/) (2023)
- [Guiding AMR Parsing with Reverse Graph Linearization](https://mlsanity.com/paper/2310.08860/) (2023)

## Abstract (arXiv)

The encoder-decoder framework has achieved promising process for many sequence generation tasks, such as neural machine translation and text summarization. Such a framework usually generates a sequence token by token from left to right, hence (1) this autoregressive decoding procedure is time-consuming when the output sentence becomes longer, and (2) it lacks the guidance of future context which is crucial to avoid under translation. To alleviate these issues, we propose a synchronous bidirectional sequence generation (SBSG) model which predicts its outputs from both sides to the middle simultaneously. In the SBSG model, we enable the left-to-right (L2R) and right-to-left (R2L) generation to help and interact with each other by leveraging interactive bidirectional attention network. Experiments on neural machine translation (En-De, Ch-En, and En-Ro) and text summarization tasks show that the proposed model significantly speeds up decoding while improving the generation quality compared to the autoregressive Transformer.