mlsanityData to 7 Oct 2026

LEPO: Latent Reasoning Policy Optimization for Large Language Models

arXiv 2604.17892 ↗First posted 20 Apr 2026cs.LG cs.AIcited by 4

On the map: LLM reasoning → Reinforcement learning for reasoning

Abstract (arXiv)

Recently, latent reasoning has been introduced into large language models (LLMs) to leverage rich information within a continuous space. However, without stochastic sampling, these methods inevitably collapse to deterministic inference, failing to discover diverse reasoning paths. To bridge the gap, we inject controllable stochasticity into latent reasoning via Gumbel-Softmax, restoring LLMs' exploratory capacity and enhancing their compatibility with Reinforcement Learning (RL). Building on this, we propose textbfunderlineLatent Rtextbfunderlineeasoning textbfunderlinePolicy textbfunderlineOptimization~(LEPO), a novel framework that applies RL directly to continuous latent representations. Specifically, in rollout stage, LEPO maintains stochasticity to enable diverse trajectory sampling, while in optimization stage, LEPO constructs a unified gradient estimation for both latent representations and discrete tokens. Extensive experiments show that LEPO significantly outperforms existing RL methods for discrete and latent reasoning.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 22 of its 26 references that are in the map. First posted on arXiv 20 Apr 2026.

Builds on

22 of 26 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025SofT-GRPO: Surpassing Discrete-Token LLM Reinforcement Learning via Gumbel-Reparameterized Soft-Thinking Policy Optimization9 Nov
  2. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scale18 Mar
  3. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning22 Jan

Other references · 19

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  2. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  3. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  4. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
  5. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,708
  6. 2024Training Large Language Models to Reason in a Continuous Latent Spacecited by 763
  7. 2025Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approachcited by 372
  8. 2025CODI: Compressing Chain-of-Thought into Continuous Space via Self-Distillationcited by 216
  9. 2024Compressed Chain of Thought: Efficient Reasoning Through Dense Representationscited by 192
  10. 2025Scaling Latent Reasoning via Looped Language Modelscited by 145
  11. 2025Soft Thinking: Unlocking the Reasoning Potential of LLMs in Continuous Concept Spacecited by 109
  12. 2025Think Silently, Think Fast: Dynamic Latent Compression of LLM Reasoning Chainscited by 81
7 more
  1. 2025Reasoning by Superposition: A Theoretical Perspective on Chain of Continuous Thoughtcited by 73
  2. 2025LLMs are Single-threaded Reasoners: Demystifying the Working Mechanism of Soft Thinkingcited by 34
  3. 2025LLM Pretraining with Continuous Conceptscited by 33
  4. 2025Soft Tokens, Hard Truthscited by 31
  5. 2025Hybrid Latent Reasoning via Reinforcement Learningcited by 30
  6. 2025Text Generation Beyond Discrete Token Samplingcited by 21
  7. 2025SwiReasoning: Switch-Thinking in Latent and Explicit for Pareto-Superior Reasoning LLMscited by 20

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026Latent Reasoning with Normalizing Flowscited by 4
  2. 2026Dropout-GRPO: Variational Stochasticity for Continuous Latent Reasoningcited by 0
  3. 2026Next Thoughts Are Distributions: Generative Autoregressive Reasoning in the Latent Spacecited by 0
  4. 2026Soft Spatial Reasoningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026TARPO: Token-Wise Latent-Explicit Reasoning via Action-Routing Policy Optimization
  2. 2026Latent-GRPO: Group Relative Policy Optimization for Latent Reasoning
  3. 2025Slow-Fast Policy Optimization: Reposition-Before-Update for LLM Reasoning
  4. 2025Thinking on the Fly: Test-Time Reasoning Enhancement via Latent Thought Policy Optimization
  5. 2025SRPO: A Cross-Domain Implementation of Large-Scale Reinforcement Learning on LLM
  6. 2025Predictive Scaling Laws for Efficient GRPO Training of Large Reasoning Models
  7. 2025Token-Efficient RL for LLM Reasoning
  8. 2025RiskPO: Risk-based Policy Optimization via Verifiable Reward for LLM Post-Training