# Retrieval-Augmented Machine Translation with Unstructured Knowledge

arXiv: https://arxiv.org/abs/2412.04342 · HTML: https://mlsanity.com/paper/2412.04342/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Placed in Generation and context integration, one of 15 directions in Retrieval-augmented generation.
- Builds on 18 of its 34 references that are in the map.
- First posted on arXiv 5 Dec 2024.

## Placed in

- Retrieval-augmented generation / [Generation and context integration](https://mlsanity.com/rag/generation-and-context-integration/)

## Builds on (most cited first, 18 of 18)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Dense Passage Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/2004.04906/) (2020)
- [Mistral 7B](https://mlsanity.com/paper/2310.06825/) (2023)
- [Billion-scale similarity search with GPUs](https://mlsanity.com/paper/1702.08734/) (2017)
- [Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection](https://mlsanity.com/paper/2310.11511/) (2023)
- [LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Models](https://mlsanity.com/paper/2403.13372/) (2024)
- [Is ChatGPT a Good NLG Evaluator? A Preliminary Study](https://mlsanity.com/paper/2303.04048/) (2023)
- [Large Language Models Are State-of-the-Art Evaluators of Translation Quality](https://mlsanity.com/paper/2302.14520/) (2023)
- [Massively Multilingual Neural Machine Translation](https://mlsanity.com/paper/1903.00089/) (2019)
- [Nearest Neighbor Machine Translation](https://mlsanity.com/paper/2010.00710/) (2020)
- [CometKiwi: IST-Unbabel 2022 Submission for the Quality Estimation Shared Task](https://mlsanity.com/paper/2209.06243/) (2022)
- [A Survey on Retrieval-Augmented Text Generation](https://mlsanity.com/paper/2202.01110/) (2022)
- [Guiding Neural Machine Translation with Retrieved Translation Pieces](https://mlsanity.com/paper/1804.02559/) (2018)
- [Neural Machine Translation with Monolingual Translation Memory](https://mlsanity.com/paper/2105.11269/) (2021)
- [RA-ISF: Learning to Answer and Understand from Retrieval Augmentation via Iterative Self-Feedback](https://mlsanity.com/paper/2403.06840/) (2024)
- [Models and Datasets for Cross-Lingual Summarisation](https://mlsanity.com/paper/2202.09583/) (2022)
- [Towards Cross-Cultural Machine Translation with Retrieval-Augmented Generation from Multilingual Knowledge Graphs](https://mlsanity.com/paper/2410.14057/) (2024)
- [Improving Retrieval Augmented Neural Machine Translation by Controlling Source and Fuzzy-Match Interactions](https://mlsanity.com/paper/2210.05047/) (2022)

## Built on by (most cited first, 9 of 9)

- [R1-T1: Fully Incentivizing Translation Capability in LLMs via Reasoning Learning](https://mlsanity.com/paper/2502.19735/) (2025)
- [On the Consistency of Multilingual Context Utilization in Retrieval-Augmented Generation](https://mlsanity.com/paper/2504.00597/) (2025)
- [Test-Time Scaling of Reasoning Models for Machine Translation](https://mlsanity.com/paper/2510.06471/) (2025)
- [Towards Style Alignment in Cross-Cultural Translation](https://mlsanity.com/paper/2507.00216/) (2025)
- [ViDove: A Translation Agent System with Multimodal Context and Memory-Augmented Reasoning](https://mlsanity.com/paper/2507.07306/) (2025)
- [Liaozhai through the Looking-Glass: On Paratextual Explicitation of Culture-Bound Terms in Machine Translation](https://mlsanity.com/paper/2509.23395/) (2025)
- [Incentivizing Parametric Knowledge via Reinforcement Learning with Verifiable Rewards for Cross-Cultural Entity Translation](https://mlsanity.com/paper/2604.16881/) (2026)
- [When Metrics Reward the Worst Translations: Internalizing Cultural Reasoning for Social Media Translation Evaluation](https://mlsanity.com/paper/2609.08156/) (2026)
- [Retrieve What Helps, Only When It Helps: Agentic Reinforcement Learning for Selective Retrieval-Augmented Machine Translation](https://mlsanity.com/paper/2610.19528/) (2026)

## Abstract (arXiv)

Retrieval-augmented generation (RAG) introduces additional information to enhance large language models (LLMs). In machine translation (MT), previous work typically retrieves in-context examples from paired MT corpora, or domain-specific knowledge from knowledge graphs, to enhance MT models. However, a large amount of world knowledge is organized in unstructured documents, and might not be fully paired across different languages. In this paper, we study retrieval-augmented MT using unstructured documents. Specifically, we build RAGtrans, the first benchmark to train and evaluate LLMs' retrieval-augmented MT ability. RAGtrans contains 169K MT samples collected via GPT-4o and human translators. Besides, documents from various languages are also provided to supply the knowledge to these samples. Based on RAGtrans, we further propose a multi-task training method to teach LLMs how to use information from multilingual documents during their translation. The method uses existing multilingual corpora to create auxiliary training objectives without additional labeling requirements. Extensive experiments show that the method improves LLMs by 1.6-3.1 BLEU and 1.0-2.0 COMET scores in En-Zh, and 1.7-2.9 BLEU and 2.1-2.7 COMET scores in En-De. We also conclude the critical difficulties that current LLMs face with this task.