mlsanityData to 14 Oct 2026

Meta Back-translation

arXiv 2102.07847 ↗First posted 15 Feb 2021cs.CL cs.LGcited by 16

Not on the map. Why?

Abstract (arXiv)

Back-translation is an effective strategy to improve the performance of Neural Machine Translation~(NMT) by generating pseudo-parallel data. However, several recent works have found that better translation quality of the pseudo-parallel data does not necessarily lead to better final translation models, while lower-quality but more diverse data often yields stronger results. In this paper, we propose a novel method to generate pseudo-parallel data from a pre-trained back-translation model. Our method is a meta-learning algorithm which adapts a pre-trained back-translation model so that the pseudo-parallel data it generates would train a forward-translation model to do well on a validation set. In our evaluations in both the standard datasets WMT En-De'14 and WMT En-Fr'14, as well as a multilingual translation setting, our method leads to significant improvements over strong baselines. Our code will be made available.

Key facts. Cited by 16 papers in the map (data through 14 Oct 2026). Builds on 28 of its 37 references that are in the map. First posted on arXiv 15 Feb 2021.

Builds on

28 of 37 references.

References · 28

  1. 2017Attention Is All You Needcited by 60,361
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
  4. 2014Sequence to Sequence Learning with Neural Networkscited by 2,986
  5. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,989
  6. 2018DARTS: Differentiable Architecture Searchcited by 1,798
  7. 2020GShard: Scaling Giant Models with Conditional Computation and Automatic Shardingcited by 1,631
  8. 2015Improving Neural Machine Translation Models with Monolingual Datacited by 788
  9. 2020Meta Pseudo Labelscited by 374
  10. 2019MASS: Masked Sequence to Sequence Pre-training for Language Generationcited by 347
  11. 2019Revisiting Self-Training for Neural Sequence Generationcited by 155
  12. 2015On Using Monolingual Corpora in Neural Machine Translationcited by 150
16 more
  1. 2017Unsupervised Neural Machine Translationcited by 149
  2. 2016Dual Learning for Machine Translationcited by 148
  3. 2019Choosing Transfer Languages for Cross-Lingual Learningcited by 144
  4. 2018When and Why are Pre-trained Word Embeddings Useful for Neural Machine Translation?cited by 126
  5. 2017Online Learning Rate Adaptation with Hypergradient Descentcited by 118
  6. 2018Meta-Learning for Low-Resource Neural Machine Translationcited by 105
  7. 2019Generative Teaching Networks: Accelerating Neural Architecture Search by Learning to Generate Synthetic Training Datacited by 104
  8. 2018Rapid Adaptation of Neural Machine Translation to New Languagescited by 82
  9. 2020Balancing Training for Multilingual Neural Machine Translationcited by 72
  10. 2019Optimizing Data Usage via Differentiable Rewardscited by 48
  11. 2019compare-mt: A Tool for Holistic Comparison of Language Generation Systemscited by 43
  12. 2016Semi-Supervised Learning for Neural Machine Translationcited by 36
  13. 2020Dynamic Data Selection and Weighting for Iterative Back-Translationcited by 25
  14. 2019Multilingual Neural Machine Translation With Soft Decoupled Encodingcited by 21
  15. 2019Target Conditioned Sampling: Optimizing Data Selection for Multilingual Neural Machine Translationcited by 11
  16. 2020Selecting Backtranslated Data from Multiple Sources for Improved Neural Machine Translationcited by 7

Built on by

Cited by year

Citing papers per year: 2021: 2; 2022: 5; 2023: 3; 2024: 3; 2025: 3.

16 papers.

Sort:

By area:

  1. 2024Best Practices and Lessons Learned on Synthetic Datacited by 106
  2. 2021BERT Learns to Teach: Knowledge Distillation with Meta Learningcited by 61
  3. 2022Meta Learning for Natural Language Processing: A Surveycited by 26
  4. 2025GSO: Challenging Software Optimization Tasks for Evaluating SWE-Agentscited by 25
  5. 2022FOAM: A Follower-aware Speaker Model For Vision-and-Language Navigationcited by 22
  6. 2025Enhancing LLM Safety Through a Theoretical Minimax Game Lenscited by 19
  7. 2021Learning to Perturb Word Embeddings for Out-of-distribution QAcited by 13
  8. 2022Learning to Generalize to More: Continuous Semantic Augmentation for Neural Machine Translationcited by 13
  9. 2022A Feature-space Multimodal Data Augmentation Technique for Text-video Retrievalcited by 13
  10. 2023Back Translation for Speech-to-text Translation Without Transcriptscited by 9
  11. 2025Overcoming Data Scarcity in Generative Language Modelling for Low-Resource Languages: A Systematic Reviewcited by 8
  12. 2023Text Style Transfer Back-Translationcited by 2
4 more
  1. 2024Asynchronous and Segmented Bidirectional Encoding for NMTcited by 2
  2. 2022End-to-End Training for Back-Translation with Categorical Reparameterization Trickcited by 0
  3. 2023INarIG: Iterative Non-autoregressive Instruct Generation Model For Word-Level Auto Completioncited by 0
  4. 2024Reducing and Exploiting Data Augmentation Noise through Meta Reweighting Contrastive Learning for Text Classificationcited by 0