# TARPO: Token-Wise Latent-Explicit Reasoning via Action-Routing Policy Optimization

arXiv: https://arxiv.org/abs/2606.05859 · HTML: https://mlsanity.com/paper/2606.05859/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning.
- Builds on 21 of its 34 references that are in the map.
- First posted on arXiv 4 Jun 2026.

## Placed in

- LLM reasoning / [Test-time compute and efficient thinking](https://mlsanity.com/lr/test-time-compute-and-efficient-thinking/)

## Builds on (most cited first, 21 of 21)

- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge](https://mlsanity.com/paper/1803.05457/) (2018)
- [GPQA: A Graduate-Level Google-Proof Q&A Benchmark](https://mlsanity.com/paper/2311.12022/) (2023)
- [Training Large Language Models to Reason in a Continuous Latent Space](https://mlsanity.com/paper/2412.06769/) (2024)
- [Token Assorted: Mixing Latent and Text Tokens for Improved Language Model Reasoning](https://mlsanity.com/paper/2502.03275/) (2025)
- [A Survey on Latent Reasoning](https://mlsanity.com/paper/2507.06203/) (2025)
- [Reasoning Beyond Language: A Comprehensive Survey on Latent Chain-of-Thought Reasoning](https://mlsanity.com/paper/2505.16782/) (2025)
- [Efficient Reasoning with Hidden Thinking](https://mlsanity.com/paper/2501.19201/) (2025)
- [Reinforced Latent Reasoning for LLM-based Recommendation](https://mlsanity.com/paper/2505.19092/) (2025)
- [LLMs are Single-threaded Reasoners: Demystifying the Working Mechanism of Soft Thinking](https://mlsanity.com/paper/2508.03440/) (2025)
- [Soft Tokens, Hard Truths](https://mlsanity.com/paper/2509.19170/) (2025)
- [SwiReasoning: Switch-Thinking in Latent and Explicit for Pareto-Superior Reasoning LLMs](https://mlsanity.com/paper/2510.05069/) (2025)
- [SofT-GRPO: Surpassing Discrete-Token LLM Reinforcement Learning via Gumbel-Reparameterized Soft-Thinking Policy Optimization](https://mlsanity.com/paper/2511.06411/) (2025)
- [Think-at-Hard: Dynamic Looped Transformers for Improved Reasoning](https://mlsanity.com/paper/2511.08577/) (2025)
- [Multiplex Thinking: Reasoning via Token-wise Branch-and-Merge](https://mlsanity.com/paper/2601.08808/) (2026)
- [Latent-GRPO: Group Relative Policy Optimization for Latent Reasoning](https://mlsanity.com/paper/2604.27998/) (2026)
- [ThinkRouter: Efficient Reasoning via Routing Thinking between Latent and Discrete Spaces](https://mlsanity.com/paper/2602.11683/) (2026)
- [SeLaR: Selective Latent Reasoning in Large Language Models](https://mlsanity.com/paper/2604.08299/) (2026)
- [Lightweight Latent Reasoning for Narrative Tasks](https://mlsanity.com/paper/2512.02240/) (2025)

## Abstract (arXiv)

Latent reasoning has emerged as a promising alternative to discrete Chain-of-Thought (CoT) in large language models (LLMs), enabling more expressive reasoning by operating over continuous representations. However, the inherently deterministic nature of continuous representations limits policy exploration in reinforcement learning (RL). To address this, we propose TARPO (Token-Wise Latent-Explicit Reasoning via Action-Routing Policy Optimization), a pure RL framework that adaptively switches between discrete token generation and continuous latent reasoning at each step. TARPO introduces a lightweight action head router that observes the current hidden state and samples a routing decision from a binary mode-selection space, preserving the stochasticity of discrete token sampling from the vocabulary. The LLM backbone and router are jointly optimized end-to-end with a shared group-relative advantage signal. Extensive experiments across Qwen2.5 (from 1.5B to 7B) and Llama-3.1-8B backbones demonstrate that TARPO consistently outperforms existing explicit and latent reasoning RL baselines across diverse benchmarks. Further analysis shows that TARPO learns adaptive token-wise switching behaviors while maintaining stable training dynamics. Our code is available at https://github.com/NKU-LITI/TARPO-master.