# DeepTrans: Deep Reasoning Translation via Reinforcement Learning

arXiv: https://arxiv.org/abs/2504.10187 · HTML: https://mlsanity.com/paper/2504.10187/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning.
- Builds on 25 of its 51 references that are in the map.
- First posted on arXiv 14 Apr 2025.

## Placed in

- LLM reasoning / [Reasoning benchmarks and evaluation](https://mlsanity.com/lr/reasoning-benchmarks-and-evaluation/)

## Builds on (most cited first, 25 of 25)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [GPT-4o System Card](https://mlsanity.com/paper/2410.21276/) (2024)
- [DAPO: An Open-Source LLM Reinforcement Learning System at Scale](https://mlsanity.com/paper/2503.14476/) (2025)
- [WebGPT: Browser-assisted question-answering with human feedback](https://mlsanity.com/paper/2112.09332/) (2021)
- [Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation](https://mlsanity.com/paper/1609.08144/) (2016)
- [Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning](https://mlsanity.com/paper/2503.09516/) (2025)
- [DeepSeek-V3 Technical Report](https://mlsanity.com/paper/2412.19437/) (2024)
- [Sequence Level Training with Recurrent Neural Networks](https://mlsanity.com/paper/1511.06732/) (2015)
- [From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judge](https://mlsanity.com/paper/2411.16594/) (2024)
- [Stop Overthinking: A Survey on Efficient Reasoning for Large Language Models](https://mlsanity.com/paper/2503.16419/) (2025)
- [Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Models](https://mlsanity.com/paper/2503.09567/) (2025)
- [From System 1 to System 2: A Survey of Reasoning Large Language Models](https://mlsanity.com/paper/2502.17419/) (2025)
- [REINFORCE++: Stabilizing Critic-Free Policy Optimization with Global Advantage Normalization](https://mlsanity.com/paper/2501.03262/) (2025)
- [Marco-o1: Towards Open Reasoning Models for Open-Ended Solutions](https://mlsanity.com/paper/2411.14405/) (2024)
- [o1-Coder: an o1 Replication for Coding](https://mlsanity.com/paper/2412.00154/) (2024)
- [All Roads Lead to Likelihood: The Value of Reinforcement Learning in Fine-Tuning](https://mlsanity.com/paper/2503.01067/) (2025)
- [DeepRAG: Thinking to Retrieve Step by Step for Large Language Models](https://mlsanity.com/paper/2502.01142/) (2025)
- [Evaluating o1-Like LLMs: Unlocking Reasoning for Translation through Comprehensive Analysis](https://mlsanity.com/paper/2502.11544/) (2025)
- [A Survey on LLM Test-Time Compute via Search: Tasks, LLM Profiling, Search Algorithms, and Relevant Frameworks](https://mlsanity.com/paper/2501.10069/) (2025)
- [R1-T1: Fully Incentivizing Translation Capability in LLMs via Reasoning Learning](https://mlsanity.com/paper/2502.19735/) (2025)
- [New Trends for Modern Machine Translation with Large Reasoning Models](https://mlsanity.com/paper/2503.10351/) (2025)
- [DeepSeek-R1 vs. o3-mini: How Well can Reasoning LLMs Evaluate MT and Summarization?](https://mlsanity.com/paper/2504.08120/) (2025)

## Built on by (most cited first, 8 of 8)

- [Think Natively: Unlocking Multilingual Reasoning with Consistency-Enhanced Reinforcement Learning](https://mlsanity.com/paper/2510.07300/) (2025)
- [How Well Do Large Reasoning Models Translate? A Comprehensive Evaluation for Multi-Domain Machine Translation](https://mlsanity.com/paper/2505.19987/) (2025)
- [TACTIC: Translation Agents with Cognitive-Theoretic Interactive Collaboration](https://mlsanity.com/paper/2506.08403/) (2025)
- [Test-Time Scaling of Reasoning Models for Machine Translation](https://mlsanity.com/paper/2510.06471/) (2025)
- [LLM Reasoning for Machine Translation: Synthetic Data Generation over Thinking Tokens](https://mlsanity.com/paper/2510.11919/) (2025)
- [Improving Heart-Focused Medical Question Answering in LLMs via Variance-Aware Rubric Rewards with GRPO](https://mlsanity.com/paper/2606.05174/) (2026)
- [LatentMT: Machine Translation with Latent Reasoning](https://mlsanity.com/paper/2607.18618/) (2026)
- [Reasoning Before Translation: Enhancing Legal Machine Translation with Structured Reasoning](https://mlsanity.com/paper/2607.19181/) (2026)

## Abstract (arXiv)

Recently, deep reasoning LLMs (e.g., OpenAI o1 and DeepSeek-R1) have shown promising performance in various downstream tasks. Free translation is an important and interesting task in the multilingual world, which requires going beyond word-for-word translation. However, the task is still under-explored in deep reasoning LLMs. In this paper, we introduce DeepTrans, a deep reasoning translation model that learns free translation via reinforcement learning (RL). Specifically, we carefully build a reward model with pre-defined scoring criteria on both the translation results and the thought processes. The reward model teaches DeepTrans how to think and free-translate the given sentences during RL. Besides, our RL training does not need any labeled translations, avoiding the human-intensive annotation or resource-intensive data synthesis. Experimental results show the effectiveness of DeepTrans. Using Qwen2.5-7B as the backbone, DeepTrans improves performance by 16.3% in literature translation, and outperforms strong deep reasoning LLMs. Moreover, we summarize the failures and interesting findings during our RL exploration. We hope this work could inspire other researchers in free translation.