mlsanityData to 7 Oct 2026

Duplex Sequence-to-Sequence Learning for Reversible Machine Translation

arXiv 2105.03458 ↗First posted 7 May 2021cs.CLcited by 9

Not on the map. Why?

Abstract (arXiv)

Sequence-to-sequence learning naturally has two directions. How to effectively utilize supervision signals from both directions? Existing approaches either require two separate models, or a multitask-learned model but with inferior performance. In this paper, we propose REDER (Reversible Duplex Transformer), a parameter-efficient model and apply it to machine translation. Either end of REDER can simultaneously input and output a distinct language. Thus REDER enables reversible machine translation by simply flipping the input and output ends. Experiments verify that REDER achieves the first success of reversible machine translation, which helps outperform its multitask-trained baselines by up to 1.3 BLEU.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 43 of its 54 references that are in the map. First posted on arXiv 7 May 2021.

Builds on

43 of 54 references.

References · 43

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
  3. 2014Sequence to Sequence Learning with Neural Networkscited by 2,974
  4. 2016Density estimation using Real NVPcited by 2,029
  5. 2020Reformer: The Efficient Transformercited by 1,658
  6. 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,486
  7. 2018Self-Attention with Relative Position Representationscited by 1,112
  8. 2014NICE: Non-linear Independent Components Estimationcited by 1,052
  9. 2015Improving Neural Machine Translation Models with Monolingual Datacited by 784
  10. 2016Sequence-Level Knowledge Distillationcited by 761
  11. 2017Masked Autoregressive Flow for Density Estimationcited by 680
  12. 2016Improving Variational Inference with Inverse Autoregressive Flowcited by 636
31 more
  1. 2017Convolutional Sequence to Sequence Learningcited by 550
  2. 2016Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translationcited by 467
  3. 2017Non-Autoregressive Neural Machine Translationcited by 457
  4. 2020Improving Massively Multilingual Neural Machine Translation and Zero-Shot Translationcited by 277
  5. 2019Massively Multilingual Neural Machine Translation in the Wild: Findings and Challengescited by 253
  6. 2017The Reversible Residual Network: Backpropagation Without Storing Activationscited by 215
  7. 2019Levenshtein Transformercited by 197
  8. 2018Deterministic Non-Autoregressive Neural Sequence Modeling by Iterative Refinementcited by 194
  9. 2016Dual Learning for Machine Translationcited by 146
  10. 2020Is MAP Decoding All You Need? The Inadequacy of the Mode in Neural Machine Translationcited by 137
  11. 2019Understanding Knowledge Distillation in Non-autoregressive Machine Translationcited by 121
  12. 2019On NMT Search Errors and Model Errors: Cat Got Your Tongue?cited by 114
  13. 2020Glancing Transformer for Non-Autoregressive Neural Machine Translationcited by 103
  14. 2020Fully Non-autoregressive Neural Machine Translation: Tricks of the Tradecited by 99
  15. 2020Non-Autoregressive Machine Translation with Latent Alignmentscited by 96
  16. 2019FlowSeq: Non-Autoregressive Conditional Sequence Generation with Generative Flowcited by 89
  17. 2018End-to-End Non-Autoregressive Neural Machine Translation with Connectionist Temporal Classificationcited by 72
  18. 2018Unsupervised Text Style Transfer using Language Models as Discriminatorscited by 70
  19. 2020A Probabilistic Formulation of Unsupervised Text Style Transfercited by 70
  20. 2019Latent-Variable Non-Autoregressive Neural Machine Translation with Deterministic Inference Using a Delta Posteriorcited by 58
  21. 2019Non-Autoregressive Machine Translation with Auxiliary Regularizationcited by 52
  22. 2016Semi-Supervised Learning for Neural Machine Translationcited by 36
  23. 2019Imitation Learning for Non-Autoregressive Neural Machine Translationcited by 36
  24. 2017Dual Supervised Learningcited by 33
  25. 2020Semi-Autoregressive Training Improves Mask-Predict Decodingcited by 33
  26. 2019Hint-Based Training for Non-Autoregressive Machine Translationcited by 28
  27. 2018Reversible Recurrent Neural Networkscited by 22
  28. 2019KERMIT: Generative Insertion-Based Modeling for Sequencescited by 21
  29. 2019Reversible GANs for Memory-efficient Image-to-Image Translationcited by 19
  30. 2019Non-autoregressive Transformer by Position Learningcited by 17
  31. 2015Agreement-based Joint Training for Bidirectional Attention-based Neural Machine Translationcited by 4

Built on by

Cited by year

Citing papers per year: 2021: 1; 2022: 5; 2023: 2; 2024: 0; 2025: 1.

9 papers.

  1. 2022Speculative Decoding: Exploiting Speculative Execution for Accelerating Seq2seq Generationcited by 169
  2. 2021Non-Autoregressive Translation with Layer-Wise Prediction and Deep Supervisioncited by 40
  3. 2023Reversible Vision Transformerscited by 33
  4. 2022One Reference Is Not Enough: Diverse Distillation with Reference Selection for Non-Autoregressive Translationcited by 20
  5. 2022Non-Monotonic Latent Alignments for CTC-Based Non-Autoregressive Machine Translationcited by 16
  6. 2022Viterbi Decoding of Directed Acyclic Transformer for Non-Autoregressive Machine Translationcited by 13
  7. 2025R3Mem: Bridging Memory Retention and Retrieval via Reversible Compressioncited by 12
  8. 2023Duplex Diffusion Models Improve Speech-to-Speech Translationcited by 5
  9. 2022Rephrasing the Reference for Non-Autoregressive Machine Translationcited by 3