mlsanityData to 14 Oct 2026

Beyond the Sentence: A Survey on Context-Aware Machine Translation with Large Language Models

arXiv 2506.07583 ↗First posted 9 Jun 2025cs.CL cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Despite the popularity of the large language models (LLMs), their application to machine translation is relatively underexplored, especially in context-aware settings. This work presents a literature review of context-aware translation with LLMs. The existing works utilise prompting and fine-tuning approaches, with few focusing on automatic post-editing and creating translation agents for context-aware machine translation. We observed that the commercial LLMs (such as ChatGPT and Tower LLM) achieved better results than the open-source LLMs (such as Llama and Bloom LLMs), and prompt-based approaches serve as good baselines to assess the quality of translations. Finally, we present some interesting future directions to explore.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 64 of its 94 references that are in the map. First posted on arXiv 9 Jun 2025.

Builds on

64 of 94 references.

References · 64

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  4. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  5. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  6. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,548
  7. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,915
  8. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
  9. 2014Sequence to Sequence Learning with Neural Networkscited by 2,986
  10. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,998
  11. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,984
  12. 2022No Language Left Behind: Scaling Human-Centered Machine Translationcited by 1,312
38 more
  1. 2020Multilingual Denoising Pre-training for Neural Machine Translationcited by 1,035
  2. 2021P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and Taskscited by 689
  3. 2023Baichuan 2: Open Large-scale Language Modelscited by 670
  4. 2024Contrastive Preference Optimization: Pushing the Boundaries of LLM Performance in Machine Translationcited by 394
  5. 2023How Good Are GPT Models at Machine Translation? A Comprehensive Evaluationcited by 348
  6. 2021Experts, Errors, and Context: A Large-Scale Study of Human Evaluation for Machine Translationcited by 338
  7. 2017Six Challenges for Neural Machine Translationcited by 335
  8. 2023Efficient and Effective Text Encoding for Chinese LLaMA and Alpacacited by 305
  9. 2023Prompting Large Language Model for Machine Translation: A Case Studycited by 250
  10. 2016Understanding Neural Networks through Representation Erasurecited by 224
  11. 2024Tower: An Open Multilingual Large Language Model for Translation-Related Taskscited by 206
  12. 2020Is MAP Decoding All You Need? The Inadequacy of the Mode in Neural Machine Translationcited by 138
  13. 2021High Quality Rather than High Model Probability: Minimum Bayes Risk Decoding with Neural Metricscited by 104
  14. 2018Context-Aware Neural Machine Translation Learns Anaphora Resolutioncited by 97
  15. 2017Neural Machine Translation with Extended Contextcited by 95
  16. 2017Evaluating Discourse Phenomena in Neural Machine Translationcited by 87
  17. 2023Large language models effectively leverage document-level context for literary translation, but critical errors persistcited by 80
  18. 2018Improving the Transformer Translation Model with Document-Level Contextcited by 74
  19. 2023Investigating the Translation Performance of a Large Multilingual Language Model: the Case of BLOOMcited by 59
  20. 2024Adapting Large Language Models for Document-Level Machine Translationcited by 59
  21. 2024From LLM to NMT: Advancing Low-Resource Machine Translation with Claudecited by 58
  22. 2019Microsoft Translator at WMT 2019: Towards Large-Scale Document-Level Neural Machine Translationcited by 56
  23. 2024(Perhaps) Beyond Human Translation: Harnessing Multi-Agent Collaboration for Translating Ultra-Long Literary Textscited by 49
  24. 2021G-Transformer for Document-level Machine Translationcited by 46
  25. 2019When and Why is Document-level Context Useful in Neural Machine Translation?cited by 45
  26. 2022Exploring Document-Level Literary Machine Translation with Parallel Paragraphs from World Literaturecited by 45
  27. 2023Explaining How Transformers Use Context to Build Predictionscited by 43
  28. 2017Document Context Neural Machine Translation with Memory Networkscited by 41
  29. 2020Does Multi-Encoder Help? A Case Study on Context-Aware Neural Machine Translationcited by 34
  30. 2024Translating Step-by-Step: Decomposing the Translation Process for Improved Translation Quality of Long-Form Textscited by 31
  31. 2020Long-Short Term Masking Transformer: A Simple but Effective Baseline for Document-level Neural Machine Translationcited by 27
  32. 2023Escaping the sentence-level paradigm in machine translationcited by 27
  33. 2024DelTA: An Online Document-Level Translation Agent Based on Multi-Level Memorycited by 27
  34. 2020Diving Deep into Context-Aware Neural Machine Translationcited by 25
  35. 2024Efficiently Exploring Large Language Models for Document-Level Machine Translation with In-context Learningcited by 23
  36. 2023Contextual Refinement of Translations: Large Language Models for Sentence and Document-Level Post-Editingcited by 21
  37. 2023Domain Terminology Integration into Machine Translation: Leveraging Large Language Modelscited by 17
  38. 2021Diverse Pretrained Context Encodings Improve Document Translationcited by 13

50 most cited of 64 All 64 (JSON)

Built on by

Not cited yet.