# Translate-R1: Cost-Aware Translation Tool Use via Reinforcement Learning

arXiv: https://arxiv.org/abs/2606.06835 · HTML: https://mlsanity.com/paper/2606.06835/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use.
- Builds on 28 of its 53 references that are in the map.
- First posted on arXiv 5 Jun 2026.

## Placed in

- LLM agents and tool use / [General LLM agent frameworks and benchmarks](https://mlsanity.com/agents/general-llm-agent-frameworks-and-benchmarks/)

## Builds on (most cited first, 28 of 28)

- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [DAPO: An Open-Source LLM Reinforcement Learning System at Scale](https://mlsanity.com/paper/2503.14476/) (2025)
- [Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning](https://mlsanity.com/paper/2503.09516/) (2025)
- [Understanding R1-Zero-Like Training: A Critical Perspective](https://mlsanity.com/paper/2503.20783/) (2025)
- [No Language Left Behind: Scaling Human-Centered Machine Translation](https://mlsanity.com/paper/2207.04672/) (2022)
- [RouteLLM: Learning to Route LLMs with Preference Data](https://mlsanity.com/paper/2406.18665/) (2024)
- [Language Models are Multilingual Chain-of-Thought Reasoners](https://mlsanity.com/paper/2210.03057/) (2022)
- [ReTool: Reinforcement Learning for Strategic Tool Use in LLMs](https://mlsanity.com/paper/2504.11536/) (2025)
- [XL-Sum: Large-Scale Multilingual Abstractive Summarization for 44 Languages](https://mlsanity.com/paper/2106.13822/) (2021)
- [ToRA: A Tool-Integrated Reasoning Agent for Mathematical Problem Solving](https://mlsanity.com/paper/2309.17452/) (2023)
- [Global MMLU: Understanding and Addressing Cultural and Linguistic Biases in Multilingual Evaluation](https://mlsanity.com/paper/2412.03304/) (2024)
- [ToRL: Scaling Tool-Integrated RL](https://mlsanity.com/paper/2503.23383/) (2025)
- [Not All Languages Are Created Equal in LLMs: Improving Multilingual Capability by Cross-Lingual-Thought Prompting](https://mlsanity.com/paper/2305.07004/) (2023)
- [Cross-lingual Prompting: Improving Zero-shot Chain-of-Thought Reasoning across Languages](https://mlsanity.com/paper/2310.14799/) (2023)
- [Acting Less is Reasoning More! Teaching Model to Act Efficiently](https://mlsanity.com/paper/2504.14870/) (2025)
- [A Survey of Multilingual Reasoning in Language Models](https://mlsanity.com/paper/2502.09457/) (2025)
- [StepTool: Enhancing Multi-Step Tool Usage in LLMs via Step-Grained Reinforcement Learning](https://mlsanity.com/paper/2410.07745/) (2024)
- [To Code or not to Code? Adaptive Tool Integration for Math Language Models via Expectation-Maximization](https://mlsanity.com/paper/2502.00691/) (2025)
- [Think Natively: Unlocking Multilingual Reasoning with Consistency-Enhanced Reinforcement Learning](https://mlsanity.com/paper/2510.07300/) (2025)
- [Enhancing Non-English Capabilities of English-Centric Large Language Models through Deep Supervision Fine-Tuning](https://mlsanity.com/paper/2503.01275/) (2025)
- [X-WebAgentBench: A Multilingual Interactive Web Benchmark for Evaluating Global Agentic System](https://mlsanity.com/paper/2505.15372/) (2025)
- [Beyond English-Centric Training: How Reinforcement Learning Improves Cross-Lingual Reasoning in LLMs](https://mlsanity.com/paper/2509.23657/) (2025)
- [TAPO: Translation Augmented Policy Optimization for Multilingual Mathematical Reasoning](https://mlsanity.com/paper/2603.25419/) (2026)
- [Think When Needed: Model-Aware Reasoning Routing for LLM-based Ranking](https://mlsanity.com/paper/2601.18146/) (2026)
- [mAceReason-Math: A Dataset of High-Quality Multilingual Math Problems Ready For RLVR](https://mlsanity.com/paper/2603.10767/) (2026)
- [Language as a Latent Variable for Reasoning Optimization](https://mlsanity.com/paper/2604.21593/) (2026)
- [Learning When to Translate for Multilingual Reasoning](https://mlsanity.com/paper/2606.02465/) (2026)

## Abstract (arXiv)

The performance gap across languages in LLMs is well documented, and closing it natively requires pretraining or fine-tuning on corpora that, for most languages, are quite limited. Translation offers an alternative: converting an input into the model's dominant language unlocks its full capabilities at once. But translating every input is wasteful for languages the model already handles, while leaving the choice to the model fails too, as LLMs are overconfident and skip the tool even when they cannot understand the input. Prior work resolves this with language-specific rules, heuristics, or external routers, each requiring manual engineering. We instead learn a single policy that decides when to translate from reward alone, developing language- and domain-adaptive introspection that invokes translation only when it cannot solve a task natively. Using data from our answer-preserving translation pipeline, we continue RL on the post-trained Qwen3-4B across 22 languages in 3 resource tiers (High, Low, XLow) and 5 domains, and introduce confidence-gated GSPO for cost-sensitive tool use. The gated policy lifts reward over the untrained baseline by +4.6 on High, +23.5 on Low, and +17.5 on XLow, and extends zero-shot to 9 held-out languages. Against an unconstrained policy that almost always translates (the reward upper bound), it preserves full reward at 66% of the cost, outperforming other cost penalties by up to +24.3 on the low-resource tiers while matching the free policy's reward on high-resource tiers at a fraction of the tool use. On 2 synthetic languages with zero prior exposure it correctly learns to always translate, improving +18.7 over the overconfident baseline.