mlsanityData to 7 Oct 2026

Evaluating o1-Like LLMs: Unlocking Reasoning for Translation through Comprehensive Analysis

arXiv 2502.11544 ↗First posted 17 Feb 2025cs.CLcited by 26

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

The o1-Like LLMs are transforming AI by simulating human cognitive processes, but their performance in multilingual machine translation (MMT) remains underexplored. This study examines: (1) how o1-Like LLMs perform in MMT tasks and (2) what factors influence their translation quality. We evaluate multiple o1-Like LLMs and compare them with traditional models like ChatGPT and GPT-4o. Results show that o1-Like LLMs establish new multilingual translation benchmarks, with DeepSeek-R1 surpassing GPT-4o in contextless tasks. They demonstrate strengths in historical and cultural translation but exhibit a tendency for rambling issues in Chinese-centric outputs. Further analysis reveals three key insights: (1) High inference costs and slower processing speeds make complex translation tasks more resource-intensive. (2) Translation quality improves with model size, enhancing commonsense reasoning and cultural translation. (3) The temperature parameter significantly impacts output quality-lower temperatures yield more stable and accurate translations, while higher temperatures reduce coherence and precision.

Key facts. Cited by 26 papers in the map (data through 7 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 22 of its 34 references that are in the map. First posted on arXiv 17 Feb 2025.

Builds on

22 of 34 references.

References · 22

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,790
  4. 2020BLEURT: Learning Robust Metrics for Text Generationcited by 1,196
  5. 2023Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debatecited by 995
  6. 2021The FLORES-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translationcited by 556
  7. 2023How Good Are GPT Models at Machine Translation? A Comprehensive Evaluationcited by 344
  8. 2024Marco-o1: Towards Open Reasoning Models for Open-Ended Solutionscited by 142
  9. 2023Towards Making the Most of ChatGPT for Machine Translationcited by 138
  10. 2023Hallucinations in Large Multilingual Translation Modelscited by 126
  11. 2024O1 Replication Journey -- Part 2: Surpassing O1-preview through Simple Distillation, Big Progress or Bitter Lesson?cited by 90
  12. 2023LLMs cannot find reasoning errors, but can correct them given the error locationcited by 85
10 more
  1. 2023Large language models effectively leverage document-level context for literary translation, but critical errors persistcited by 80
  2. 2022On the Effect of Pretraining Corpora on In-context Learning by a Large-scale Language Modelcited by 78
  3. 2024(Perhaps) Beyond Human Translation: Harnessing Multi-Agent Collaboration for Translating Ultra-Long Literary Textscited by 49
  4. 2023Empowering Cross-lingual Abilities of Instruction-tuned Large Language Models by Translation-following demonstrationscited by 34
  5. 2022Using natural language prompts for machine translationcited by 27
  6. 2023Domain Terminology Integration into Machine Translation: Leveraging Large Language Modelscited by 17
  7. 2024DUAL-REFLECT: Enhancing Large Language Models for Reflective Translation through Dual Learning Feedback Mechanismscited by 16
  8. 2023Improving Machine Translation with Large Language Models: A Preliminary Study with Cooperative Decodingcited by 10
  9. 2023Towards Effective Disambiguation for Machine Translation with Large Language Modelscited by 8
  10. 2024LLM-based Translation Inference with Iterative Bilingual Understandingcited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 19; 2026: 7.

26 papers.

Sort:

By area:

  1. 2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Modelscited by 381
  2. 2025Inverse Scaling in Test-Time Computecited by 49
  3. 2025MT-R1-Zero: Advancing LLM-based Machine Translation via R1-Zero-like Reinforcement Learningcited by 48
  4. 2025Generative to Agentic AI: Survey, Conceptualization, and Challengescited by 32
  5. 2025Trade-offs in Large Reasoning Models: An Empirical Analysis of Deliberative and Adaptive Reasoning over Foundational Capabilitiescited by 20
  6. 2025New Trends for Modern Machine Translation with Large Reasoning Modelscited by 17
  7. 2025SSR-Zero: Simple Self-Rewarding Reinforcement Learning for Machine Translationcited by 13
  8. 2025ExTrans: Multilingual Deep Reasoning Translation via Exemplar-Enhanced Reinforcement Learningcited by 11
  9. 2025DeepTrans: Deep Reasoning Translation via Reinforcement Learningcited by 8
  10. 2025Self-Route: Automatic Mode Switching via Capability Estimation for Efficient Reasoningcited by 8
  11. 2026The Hot Mess of AI: How Does Misalignment Scale With Model Intelligence and Task Complexity?cited by 8
  12. 2025Reasoning or Not? A Comprehensive Evaluation of Reasoning LLMs for Dialogue Summarizationcited by 6
14 more
  1. 2025How Well Do Large Reasoning Models Translate? A Comprehensive Evaluation for Multi-Domain Machine Translationcited by 5
  2. 2025Beyond Policy Optimization: A Data Curation Flywheel for Sparse-Reward Long-Horizon Planningcited by 4
  3. 2025LLM Reasoning for Machine Translation: Synthetic Data Generation over Thinking Tokenscited by 4
  4. 2025NiuTrans.LMT: Toward Inclusive and Scalable Multilingual Machine Translation with LLMscited by 4
  5. 2025TAT-R1: Terminology-Aware Translation with Reinforcement Learning and Word Alignmentcited by 3
  6. 2025KROMA: Ontology Matching with Knowledge Retrieval and Large Language Modelscited by 2
  7. 2026GRRM: Group Relative Reward Modeling for Machine Translationcited by 2
  8. 2025Ready to Translate, Not to Represent? Bias and Performance Gaps in Multilingual LLMs Across Language Families and Domainscited by 1
  9. 2025Can Peter Pan Survive MT? A Stylometric Study of LLMs, NMTs, and HTs in Children's Literature Translationcited by 0
  10. 2026Loong: A Human-Like Long Document Translation Agent with Observe-and-Act Adaptive Context Selectioncited by 0
  11. 2026Reasoning Before Translation: Enhancing Legal Machine Translation with Structured Reasoningcited by 0
  12. 2026Translation with Thought: Difficulty-Adaptive Reasoning via Reinforcement Learning for Multi-Domain Machine Translationcited by 0
  13. 2026Reasoning about In-Context Samples for Machine-Translationcited by 0
  14. 2026When Does Reasoning Help in Machine Translation? A Hierarchical Analysis of LRM Reasoning Tracescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Please Translate Again: Two Simple Experiments on Whether Human-Like Reasoning Helps Translation
  2. 2023An Empirical Study of Instruction-tuning Large Language Models in Chinese
  3. 2025R1-T1: Fully Incentivizing Translation Capability in LLMs via Reasoning Learning
  4. 2024Enhancing Function-Calling Capabilities in LLMs: Strategies for Prompt Formats, Data Integration, and Multilingual Translation
  5. 2025LaoBench: A Large-Scale Multidimensional Lao Benchmark for Large Language Models
  6. 2024DRT: Deep Reasoning Translation via Long Chain-of-Thought
  7. 2025Adapting Language-Specific LLMs to a Reasoning Model in One Day via Model Merging -- An Open Recipe
  8. 2025Estonian WinoGrande Dataset: Comparative Analysis of LLM Performance on Human and Machine Translation