mlsanityData to 14 Oct 2026

Towards Making the Most of ChatGPT for Machine Translation

arXiv 2303.13780 ↗First posted 24 Mar 2023cs.CLcited by 138

Not on the map. Why?

Abstract (arXiv)

ChatGPT shows remarkable capabilities for machine translation (MT). Several prior studies have shown that it achieves comparable results to commercial systems for high-resource languages, but lags behind in complex tasks, e.g., low-resource and distant-language-pairs translation. However, they usually adopt simple prompts which can not fully elicit the capability of ChatGPT. In this paper, we aim to further mine ChatGPT's translation ability by revisiting several aspects: temperature, task information, and domain information, and correspondingly propose an optimal temperature setting and two (simple but effective) prompts: Task-Specific Prompts (TSP) and Domain-Specific Prompts (DSP). We show that: 1) The performance of ChatGPT depends largely on temperature, and a lower temperature usually can achieve better performance; 2) Emphasizing the task information can further improve ChatGPT's performance, particularly in complex MT tasks; 3) Introducing domain information can elicit ChatGPT's generalization ability and improve its performance in the specific domain; 4) ChatGPT tends to generate hallucinations for non-English-centric MT tasks, which can be partially addressed by our proposed prompts but still need to be highlighted for the MT/NLP community. We also explore the effects of advanced in-context learning strategies and find a (negative but interesting) observation: the powerful chain-of-thought prompt leads to word-by-word translation behavior, thus bringing significant translation degradation.

Key facts. Cited by 138 papers in the map (data through 14 Oct 2026). Builds on 30 of its 54 references that are in the map. First posted on arXiv 24 Mar 2023.

Builds on

30 of 54 references.

References · 30

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2022Training language models to follow instructions with human feedbackcited by 15,971
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  5. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  6. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  7. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
  8. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  9. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  10. 2022Emergent Abilities of Large Language Modelscited by 2,512
  11. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,799
  12. 2022No Language Left Behind: Scaling Human-Centered Machine Translationcited by 1,312
18 more
  1. 2021What Makes Good In-Context Examples for GPT-3?cited by 1,195
  2. 2022Large Language Models Are Human-Level Prompt Engineerscited by 1,045
  3. 2020COMET: A Neural Framework for MT Evaluationcited by 1,033
  4. 2022A Survey on In-context Learningcited by 721
  5. 2021The FLORES-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translationcited by 560
  6. 2023Is ChatGPT a General-Purpose Natural Language Processing Task Solver?cited by 488
  7. 2023Large Language Models Are State-of-the-Art Evaluators of Translation Qualitycited by 420
  8. 2023How Good Are GPT Models at Machine Translation? A Comprehensive Evaluationcited by 348
  9. 2017Six Challenges for Neural Machine Translationcited by 335
  10. 2023Can ChatGPT Understand Too? A Comparative Study on ChatGPT and Fine-tuned BERTcited by 155
  11. 2022Prompting PaLM for Translation: Assessing Strategies and Performancecited by 145
  12. 2019Comparison of Diverse Decoding Methods from Conditional Language Modelscited by 78
  13. 2020Self-Attention with Cross-Lingual Position Representationcited by 27
  14. 2021Progressive Multi-Granularity Training for Non-Autoregressive Translationcited by 25
  15. 2022Toward Efficient Language Model Pretraining and Downstream Adaptation via Self-Evolution: A Case Study on SuperGLUEcited by 24
  16. 2022Vega-MT: The JD Explore Academy Translation System for WMT22cited by 9
  17. 2022Bridging Cross-Lingual Gaps During Leveraging the Multilingual Sequence-to-Sequence Pretraining for Text Generation and Understandingcited by 6
  18. 2018Neural Phrase-to-Phrase Machine Translationcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 35; 2024: 60; 2025: 32; 2026: 11.

138 papers.

Sort:

By area:

  1. 2024Leak, Cheat, Repeat: Data Contamination and Evaluation Malpractices in Closed-Source LLMscited by 229
  2. 2024Mitigating Hallucinations in Large Vision-Language Models with Instruction Contrastive Decodingcited by 203
  3. 2023Multilingual Machine Translation with Large Language Models: Empirical Results and Analysiscited by 160
  4. 2023Hallucinations in Large Multilingual Translation Modelscited by 129
  5. 2023A Systematic Study and Comprehensive Evaluation of ChatGPT on Benchmark Datasetscited by 123
  6. 2024Large Language Models Meet NLP: A Surveycited by 103
  7. 2024Revisiting Catastrophic Forgetting in Large Language Model Tuningcited by 99
  8. 2023Is ChatGPT a Highly Fluent Grammatical Error Correction System? A Comprehensive Evaluationcited by 97
  9. 2024Intention Analysis Makes LLMs A Good Jailbreak Defendercited by 74
  10. 2023Recursively Summarizing Enables Long-Term Dialogue Memory in Large Language Modelscited by 62
  11. 2024Revisiting Demonstration Selection Strategies in In-Context Learningcited by 58
  12. 2023CodeTransOcean: A Comprehensive Multilingual Benchmark for Code Translationcited by 57
38 more
  1. 2025A Survey of LLM-Driven AI Agent Communication: Protocols, Security Risks, and Defense Countermeasurescited by 57
  2. 2023ChatGPT vs Human-authored Text: Insights into Controllable Text Summarization and Sentence Style Transfercited by 50
  3. 2024Language Model Alignment in Multilingual Trolley Problemscited by 46
  4. 2023Exploring Human-Like Translation Strategy with Large Language Modelscited by 44
  5. 2024Revisiting Knowledge Distillation for Autoregressive Language Modelscited by 42
  6. 2024Learning to Ask: When LLM Agents Meet Unclear Instructioncited by 42
  7. 2024DynamicKV: Task-Aware Adaptive KV Cache Compression for Long Context LLMscited by 42
  8. 2023CHEAT: A Large-scale Dataset for Detecting ChatGPT-writtEn AbsTractscited by 40
  9. 2024TEaR: Improving LLM-based Machine Translation with Systematic Self-Refinementcited by 39
  10. 2023GPTAraEval: A Comprehensive Evaluation of ChatGPT on Arabic NLPcited by 37
  11. 2024Compress to Impress: Unleashing the Potential of Compressive Memory in Real-World Long-Term Conversationscited by 36
  12. 2024SelectIT: Selective Instruction Tuning for LLMs via Uncertainty-Aware Self-Reflectioncited by 34
  13. 2024ROSE Doesn't Do That: Boosting the Safety of Instruction-Tuned Large Language Models with Reverse Prompt Contrastive Decodingcited by 32
  14. 2023How to Design Translation Prompts for ChatGPT: An Empirical Studycited by 31
  15. 2023ChatGraph: Interpretable Text Classification by Converting ChatGPT Knowledge to Graphscited by 27
  16. 2024TasTe: Teaching Large Language Models to Translate through Self-Reflectioncited by 27
  17. 2024DelTA: An Online Document-Level Translation Agent Based on Multi-Level Memorycited by 27
  18. 2023ChatABL: Abductive Learning via Natural Language Interaction with ChatGPTcited by 26
  19. 2025Evaluating o1-Like LLMs: Unlocking Reasoning for Translation through Comprehensive Analysiscited by 26
  20. 2023ChatGPT: More than a Weapon of Mass Deception, Ethical challenges and responses from the Human-Centered Artificial Intelligence (HCAI) perspectivecited by 22
  21. 2024MQM-APE: Toward High-Quality Error Annotation Predictors with Automatic Post-Editing in LLM Translation Evaluatorscited by 22
  22. 2023Can Linguistic Knowledge Improve Multimodal Alignment in Vision-Language Pretraining?cited by 21
  23. 2024GPT-4 vs. Human Translators: A Comprehensive Evaluation of Translation Quality Across Languages, Domains, and Expertise Levelscited by 21
  24. 2025R1-T1: Fully Incentivizing Translation Capability in LLMs via Reasoning Learningcited by 21
  25. 2023BHASA: A Holistic Southeast Asian Linguistic and Cultural Evaluation Suite for Large Language Modelscited by 18
  26. 2024Healthcare Copilot: Eliciting the Power of General LLMs for Medical Consultationcited by 17
  27. 2023Chain-of-Dictionary Prompting Elicits Translation in Large Language Modelscited by 16
  28. 2023Does ChatGPT have Theory of Mind?cited by 16
  29. 2023TARJAMAT: Evaluation of Bard and ChatGPT on Machine Translation of Ten Arabic Varietiescited by 16
  30. 2023An Empirical Study of Translation Hypothesis Ensembling with Large Language Modelscited by 16
  31. 2024How to Make the Most of LLMs' Grammatical Knowledge for Acceptability Judgmentscited by 14
  32. 2025Compositional Translation: A Novel LLM-based Approach for Low-resource Machine Translationcited by 14
  33. 2023Deliberate then Generate: Enhanced Prompting Framework for Text Generationcited by 13
  34. 2024Take Care of Your Prompt Bias! Investigating and Mitigating Prompt Bias in Factual Knowledge Extractioncited by 13
  35. 2024Quite Good, but Not Enough: Nationality Bias in Large Language Models -- A Case Study of ChatGPTcited by 13
  36. 2024Uncertainty Aware Learning for Language Model Alignmentcited by 13
  37. 2023Fine-tune Language Models to Approximate Unbiased In-context Learningcited by 12
  38. 2024Machine Translation with Large Language Models: Prompt Engineering for Persian, English, and Russian Directionscited by 12

50 most cited of 138 All 138 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023Pushing the Limits of ChatGPT on NLP Tasks
  2. 2023Summary of ChatGPT-Related Research and Perspective Towards the Future of Large Language Models
  3. 2023Improving ChatGPT Prompt for Code Generation
  4. 2025Beyond Decoder-only: Large Language Models Can be Good Encoders for Machine Translation
  5. 2023A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivity
  6. 2025BatchGEMBA: Token-Efficient Machine Translation Evaluation with Batched Prompting and Prompt Compression
  7. 2025The Art of Asking: Multilingual Prompt Optimization for Synthetic Data
  8. 2025Investigating Test-Time Scaling with Reranking for Machine Translation