mlsanityData to 7 Oct 2026

Neural Machine Translation with Monolingual Translation Memory

arXiv 2105.11269 ↗First posted 24 May 2021cs.CL cs.AIcited by 61

Not on the map. Why?

Abstract (arXiv)

Prior work has proved that Translation memory (TM) can boost the performance of Neural Machine Translation (NMT). In contrast to existing work that uses bilingual corpus as TM and employs source-side similarity search for memory retrieval, we propose a new framework that uses monolingual memory and performs learnable memory retrieval in a cross-lingual manner. Our framework has unique advantages. First, the cross-lingual memory retriever allows abundant monolingual data to be TM. Second, the memory retriever and NMT model can be jointly optimized for the ultimate translation goal. Experiments show that the proposed method obtains substantial improvements. Remarkably, it even outperforms strong TM-augmented NMT baselines using bilingual TM. Owning to the ability to leverage monolingual data, our model also demonstrates effectiveness in low-resource and domain adaptation scenarios.

Key facts. Cited by 61 papers in the map (data through 7 Oct 2026). Builds on 34 of its 57 references that are in the map. First posted on arXiv 24 May 2021.

Builds on

34 of 57 references.

References · 34

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  4. 2020Language Models are Few-Shot Learnerscited by 32,199
  5. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  6. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
  7. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  8. 2017Billion-scale similarity search with GPUscited by 2,477
  9. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
  10. 2017Get To The Point: Summarization with Pointer-Generator Networkscited by 1,281
  11. 2016Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphscited by 853
  12. 2019Generalization through Memorization: Nearest Neighbor Language Modelscited by 788
22 more
  1. 2015Improving Neural Machine Translation Models with Monolingual Datacited by 784
  2. 2017Reading Wikipedia to Answer Open-Domain Questionscited by 777
  3. 2019Latent Retrieval for Weakly Supervised Open Domain Question Answeringcited by 691
  4. 2017Six Challenges for Neural Machine Translationcited by 334
  5. 2016Incorporating Copying Mechanism in Sequence-to-Sequence Learningcited by 217
  6. 2020Nearest Neighbor Machine Translationcited by 209
  7. 2020Unsupervised Domain Clusters in Pretrained Language Modelscited by 160
  8. 2019Revisiting Self-Training for Neural Sequence Generationcited by 154
  9. 2015On Using Monolingual Corpora in Neural Machine Translationcited by 149
  10. 2016Dual Learning for Machine Translationcited by 146
  11. 2017Data Augmentation for Low-Resource Neural Machine Translationcited by 139
  12. 2014Asymmetric LSH (ALSH) for Sublinear Time Maximum Inner Product Search (MIPS)cited by 109
  13. 2020Pre-training via Paraphrasingcited by 92
  14. 2017Generating Sentences by Editing Prototypescited by 88
  15. 2018Retrieve and Refine: Improved Sequence Generation Models For Dialoguecited by 82
  16. 2018A Retrieve-and-Edit Framework for Predicting Structured Outputscited by 73
  17. 2018A Survey of Domain Adaptation for Neural Machine Translationcited by 64
  18. 2018Guiding Neural Machine Translation with Retrieved Translation Piecescited by 63
  19. 2018Skeleton-to-Response: Dialogue Generation Guided by Retrieval Memorycited by 49
  20. 2018Response Generation by Context-aware Prototype Editingcited by 40
  21. 2017Memory-augmented Neural Machine Translationcited by 12
  22. 2019Learning to Reuse Translations: Guiding Neural Machine Translation with Examplescited by 5

Built on by

Cited by year

Citing papers per year: 2021: 4; 2022: 18; 2023: 15; 2024: 22; 2025: 2.

61 papers.

Sort:

By area:

  1. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,185
  2. 2023Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Modelscited by 695
  3. 2024Retrieval-Augmented Generation for AI-Generated Content: A Surveycited by 352
  4. 2022A Survey on Retrieval-Augmented Text Generationcited by 157
  5. 2023Lift Yourself Up: Retrieval-augmented Text Generation with Self Memorycited by 103
  6. 2024Retrieval-Augmented Generation for Natural Language Processing: A Surveycited by 70
  7. 2024AlphaFin: Benchmarking Financial Analysis with Retrieval-Augmented Stock-Chain Frameworkcited by 47
  8. 2024With Greater Text Comes Greater Necessity: Inference-Time Training Helps Long Text Generationcited by 36
  9. 2022On the Evaluation Metrics for Paraphrase Generationcited by 32
  10. 2024Can Language Beat Numerical Regression? Language-Based Multimodal Trajectory Predictioncited by 31
  11. 2023WebBrain: Learning to Generate Factually Correct Articles for Queries by Grounding on Large Web Corpuscited by 29
  12. 2023Copy Is All You Needcited by 25
38 more
  1. 2024ALR2: A Retrieve-then-Reason Framework for Long-context Question Answeringcited by 25
  2. 2024Jill Watson: A Virtual Teaching Assistant powered by ChatGPTcited by 24
  3. 2024Towards a Search Engine for Machines: Unified Ranking for Multiple Retrieval-Augmented Large Language Modelscited by 23
  4. 2024Leveraging Hallucinations to Reduce Manual Prompt Dependency in Promptable Segmentationcited by 20
  5. 2023A Paradigm Shift: The Future of Machine Translation Lies with Large Language Modelscited by 19
  6. 2021TranSmart: A Practical Interactive Machine Translation Systemcited by 17
  7. 2021Self-Training Sampling with Monolingual Data Uncertainty for Neural Machine Translationcited by 17
  8. 2024How Much Data is Enough Data? Fine-Tuning Large Language Models for In-House Translation: Performance Evaluation Across Multiple Dataset Sizescited by 17
  9. 2022Retrieval-Augmented Multilingual Keyphrase Generation with Retriever-Generator Iterative Trainingcited by 16
  10. 2022Towards Robust k-Nearest-Neighbor Machine Translationcited by 15
  11. 2021Transductive Learning for Unsupervised Text Style Transfercited by 12
  12. 2022Neural Machine Translation with Contrastive Translation Memoriescited by 12
  13. 2023Simple and Scalable Nearest Neighbor Machine Translationcited by 12
  14. 2024Retrieval is Accurate Generationcited by 10
  15. 2021Non-Parametric Online Learning from Human Feedback for Neural Machine Translationcited by 8
  16. 2024Retrieval-Augmented Machine Translation with Unstructured Knowledgecited by 8
  17. 2022Retrieve-and-Fill for Scenario-based Task-Oriented Semantic Parsingcited by 7
  18. 2022Nearest Neighbor Non-autoregressive Text Generationcited by 7
  19. 2022Zero-shot Domain Adaptation for Neural Machine Translation with Retrieved Phrase-level Promptscited by 7
  20. 2022Improving Retrieval Augmented Neural Machine Translation by Controlling Source and Fuzzy-Match Interactionscited by 7
  21. 2022Diverse Parallel Data Synthesis for Cross-Database Adaptation of Text-to-SQL Parserscited by 7
  22. 2022Controlling Styles in Neural Machine Translation with Activation Promptcited by 7
  23. 2024Retrieving Examples from Memory for Retrieval Augmented Neural Machine Translation: A Systematic Comparisoncited by 7
  24. 2022Integrating Translation Memories into Non-Autoregressive Machine Translationcited by 5
  25. 2022ConReader: Exploring Implicit Relations in Contracts for Contract Clause Extractioncited by 5
  26. 2022N-gram Is Back: Residual Learning of Neural Text Generation with n-gram Language Modelcited by 3
  27. 2023Decouple knowledge from parameters for plug-and-play language modelingcited by 3
  28. 2023Multilingual k-Nearest-Neighbor Machine Translationcited by 3
  29. 2024Optimizing Rare Word Accuracy in Direct Speech Translation with a Retrieval-and-Demonstration Approachcited by 3
  30. 2022Improving Robustness of Retrieval Augmented Translation via Shuffling of Suggestionscited by 2
  31. 2022Bilingual Synchronization: Restoring Translational Relationships with Editing Operationscited by 2
  32. 2022Better Datastore, Better Translation: Generating Datastores from Pre-Trained Models for Nearest Neural Machine Translationcited by 2
  33. 2023Prompting Neural Machine Translation with Translation Memoriescited by 2
  34. 2023Optimizing Factual Accuracy in Text Generation through Dynamic Knowledge Selectioncited by 2
  35. 2023Towards Example-Based NMT with Multi-Levenshtein Transformerscited by 2
  36. 2024Mafin: Enhancing Black-Box Embeddings with Model Augmented Fine-Tuningcited by 2
  37. 2024MATTER: Memory-Augmented Transformer Using Heterogeneous Knowledge Sourcescited by 2
  38. 2023Rethinking Translation Memory Augmented Neural Machine Translationcited by 1

50 most cited of 61 All 61 (JSON)