mlsanityData to 14 Oct 2026

Fine-tuning Large Language Models for Domain-specific Machine Translation

arXiv 2402.15061 ↗First posted 23 Feb 2024cs.CL cs.LGcited by 43

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) have shown great potential in domain-specific machine translation (MT). However, one major issue is that LLMs pre-trained on general domain corpus might not generalize well to specific domains due to the lack of domain-specific knowledge. To address this issue, this paper focuses on enhancing the domain-specific MT capability of LLMs, by providing high-quality training datasets and proposing a novel fine-tuning framework denoted by DragFT. DragFT augments LLMs via three techniques: (i) Dictionary-enhanced prompting integrates dictionary information into prompts to improve the translation of domain-specific terminology.; (ii) RAG-based few-shot example selection provides high-quality examples that simulate both the domain and style characteristics; (iii) Fine-tuning with few-shot examples further enhances performance when using in-domain examples. We deploy DragFT on three well-known LLM backbones with 13B training parameters to validate its effectiveness. The results on three domain-specific datasets show that DragFT achieves a significant performance boost and shows superior performance compared to advanced models such as GPT-3.5 and GPT-4o. The drastic performance improvement of DragFT over existing LLMs can be attributed to incorporating relevant knowledge while mitigating noise.

Key facts. Cited by 43 papers in the map (data through 14 Oct 2026). Builds on 32 of its 43 references that are in the map. First posted on arXiv 23 Feb 2024.

Builds on

32 of 43 references.

References · 32

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  4. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  5. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,989
  6. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,984
  7. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,818
  8. 2022No Language Left Behind: Scaling Human-Centered Machine Translationcited by 1,312
  9. 2023Baichuan 2: Open Large-scale Language Modelscited by 670
  10. 2021Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigmcited by 606
  11. 2021The FLORES-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translationcited by 560
  12. 2023How Good Are GPT Models at Machine Translation? A Comprehensive Evaluationcited by 348
20 more
  1. 2017Six Challenges for Neural Machine Translationcited by 335
  2. 2021Few-shot Learning with Multilingual Language Modelscited by 278
  3. 2020Improving Massively Multilingual Neural Machine Translation and Zero-Shot Translationcited by 277
  4. 2023Prompting Large Language Model for Machine Translation: A Case Studycited by 250
  5. 2022In-context Examples Selection for Machine Translationcited by 172
  6. 2022Prompting PaLM for Translation: Assessing Strategies and Performancecited by 145
  7. 2021Language Models are Few-shot Multilingual Learnerscited by 92
  8. 2023A Paradigm Shift in Machine Translation: Boosting Translation Performance of Large Language Modelscited by 89
  9. 2023Dictionary-based Phrase-level Prompting of Large Language Models for Machine Translationcited by 74
  10. 2023PolyLM: An Open Source Polyglot Large Language Modelcited by 64
  11. 2023Adaptive Machine Translation with Large Language Modelscited by 58
  12. 2023Eliciting the Translation Ability of Large Language Models via Multilingual Finetuning with Translation Instructionscited by 56
  13. 2023Steering Large Language Models for Machine Translation with Finetuning and In-Context Learningcited by 49
  14. 2023ParroT: Translating during Chat using Large Language Models tuned with Human Translation and Feedbackcited by 34
  15. 2023Fine-tuning Large Language Models for Adaptive Machine Translationcited by 21
  16. 2023Domain Terminology Integration into Machine Translation: Leveraging Large Language Modelscited by 17
  17. 2023Chain-of-Dictionary Prompting Elicits Translation in Large Language Modelscited by 16
  18. 2023Cross-Lingual Supervision improves Large Language Models Pre-trainingcited by 8
  19. 2023Neural Machine Translation Models Can Learn to be Few-shot Learnerscited by 8
  20. 2023Towards Effective Disambiguation for Machine Translation with Large Language Modelscited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 14; 2025: 21; 2026: 8.

43 papers.

Sort:

By area:

  1. 2024A Survey on LoRA of Large Language Modelscited by 67
  2. 2025LLM Agents for Education: Advances and Applicationscited by 59
  3. 2024LoRA Land: 310 Fine-tuned LLMs that Rival GPT-4, A Technical Reportcited by 47
  4. 2025Transformer Copilot: Learning from The Mistake Log in LLM Fine-tuningcited by 21
  5. 2024How Much Data is Enough Data? Fine-Tuning Large Language Models for In-House Translation: Performance Evaluation Across Multiple Dataset Sizescited by 17
  6. 2024Enhancing Code Translation in Language Models with Few-Shot Learning via Retrieval-Augmented Generationcited by 16
  7. 2024Towards Achieving Human Parity on End-to-end Simultaneous Speech Translation via LLM Agentcited by 14
  8. 2024Enhancing Agent Learning through World Dynamics Modelingcited by 9
  9. 2025Evaluating Test-Time Scaling LLMs for Legal Reasoning: OpenAI o1, DeepSeek-R1, and Beyondcited by 9
  10. 2025Science Across Languages: Assessing LLM Multilingual Translation of Scientific Paperscited by 8
  11. 2024Large Model Strategic Thinking, Small Model Efficiency: Transferring Theory of Mind in Large Language Modelscited by 7
  12. 2024LlamaTurk: Adapting Open-Source Generative Large Language Models for Low-Resource Languagecited by 6
31 more
  1. 2024Boosting LLM Translation Skills without General Ability Loss via Rationale Distillationcited by 6
  2. 2025How Well Do Large Reasoning Models Translate? A Comprehensive Evaluation for Multi-Domain Machine Translationcited by 5
  3. 2024Instruction-tuned Large Language Models for Machine Translation in the Medical Domaincited by 4
  4. 2024All-in-One Tuning and Structural Pruning for Domain-Specific LLMscited by 4
  5. 2025Robot Operation of Home Appliances by Reading User Manualscited by 4
  6. 2025Aligning Knowledge Graphs and Language Models for Factual Accuracycited by 4
  7. 2024Simultaneous Interpretation Corpus Construction by Large Language Models in Distant Language Paircited by 3
  8. 2025Large Language Models for Bioinformaticscited by 3
  9. 2025DMDTEval: An Evaluation and Analysis of LLMs on Disambiguation in Multi-domain Translationcited by 3
  10. 2025TAT-R1: Terminology-Aware Translation with Reinforcement Learning and Word Alignmentcited by 3
  11. 2025Translation Analytics for Freelancers: I. Introduction, Data Preparation, Baseline Evaluationscited by 2
  12. 2025ELR-1000: A Community-Generated Dataset for Endangered Indic Indigenous Languagescited by 2
  13. 2026Finding the Translation Switch: Discovering and Exploiting the Task-Initiation Features in LLMscited by 2
  14. 2024From Generalist to Specialist: Improving Large Language Models for Medical Physics Using ARCoTcited by 1
  15. 2025Privacy-Preserving Transformers: SwiftKey's Differential Privacy Implementationcited by 1
  16. 2025Position: Restructuring of Categories and Implementation of Guidelines Essential for VLM Adoption in Healthcarecited by 1
  17. 2025A Knowledge Graph and a Tripartite Evaluation Framework Make Retrieval-Augmented Generation Scalable and Transparentcited by 1
  18. 2025Synergy over Discrepancy: A Partition-Based Approach to Multi-Domain LLM Fine-Tuningcited by 1
  19. 2026Self Knowledge Re-expression: A Fully Local Method for Adapting LLMs to Tasks Using Intrinsic Knowledgecited by 1
  20. 2024LIMBA: An Open-Source Framework for the Preservation and Valorization of Low-Resource Languages using Generative Modelscited by 0
  21. 2025Optimizing GPT for Video Understanding: Zero-Shot Performance and Prompt Engineeringcited by 0
  22. 2025ReCellTy: Domain-Specific Knowledge Graph Retrieval-Augmented LLMs Reasoning Workflow for Single-Cell Annotationcited by 0
  23. 2025FuxiMT: Sparsifying Large Language Models for Chinese-Centric Multilingual Machine Translationcited by 0
  24. 2025Collaborative Editable Modelcited by 0
  25. 2025Data-Efficient Domain Adaptation for LLM-based MT using Contrastive Preference Optimizationcited by 0
  26. 2026How Does Prefix Matter in Reasoning Model Tuning?cited by 0
  27. 2026Consensus-Aligned Neuron Efficient Fine-Tuning Large Language Models for Multi-Domain Machine Translationcited by 0
  28. 2026EVGeoQA: Benchmarking LLMs on Dynamic, Multi-Objective Geo-Spatial Explorationcited by 0
  29. 2026Translation Analytics for Freelancers II: Benchmarking Local LLMs for Confidential Translation Workflowscited by 0
  30. 2026PAMT: Process-Aligned Reinforcement Learning for Multi-Domain Machine Translationcited by 0
  31. 2026Training-Free Token-Level Steering for LLM Personalized Co-Writingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Large Language Model for Multi-Domain Translation: Benchmarking and Domain CoT Fine-tuning
  2. 2023Towards Better Parameter-Efficient Fine-Tuning for Large Language Models: A Position Paper
  3. 2024A Preference-driven Paradigm for Enhanced Translation with Large Language Models
  4. 2024Domain-Specific Translation with Open-Source Large Language Models: Resource-Oriented Analysis
  5. 2025FineScope : SAE-guided Data Selection Enables Domain Specific LLM Pruning and Finetuning
  6. 2025Bridging the Linguistic Divide: A Survey on Leveraging Large Language Models for Machine Translation
  7. 2024Fine-tuning large language models for domain adaptation: Exploration of training strategies, scaling, model merging and synergistic capabilities
  8. 2024An Empirical Study on Parameter-Efficient Fine-Tuning for MultiModal Large Language Models