mlsanityData to 14 Oct 2026

Variational Reasoning for Language Models

arXiv 2509.22637 ↗First posted 26 Sep 2025cs.CL cs.AI cs.LGcited by 8

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

We introduce a variational reasoning framework for language models that treats thinking traces as latent variables and optimizes them through variational inference. Starting from the evidence lower bound (ELBO), we extend it to a multi-trace objective for tighter bounds and propose a forward-KL formulation that stabilizes the training of the variational posterior. We further show that rejection sampling finetuning and binary-reward RL, including GRPO, can be interpreted as local forward-KL objectives, where an implicit weighting by model accuracy naturally arises from the derivation and reveals a previously unnoticed bias toward easier questions. We empirically validate our method on the Qwen 2.5 and Qwen 3 model families across a wide range of reasoning tasks. Overall, our work provides a principled probabilistic perspective that unifies variational inference with RL-style methods and yields stable objectives for improving the reasoning ability of language models. Our code is available at https://github.com/sail-sg/variational-reasoning.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 53 of its 78 references that are in the map. First posted on arXiv 26 Sep 2025.

Builds on

53 of 78 references.

Closest ancestors · 5

Same direction, newest first

  1. 2025Optimizing Anytime Reasoning via Budget Relative Policy Optimization19 May
  2. 2025Reasoning to Learn from Latent Thoughts24 Mar
  3. 2025s1: Simple test-time scaling31 Jan
  4. 2025BRiTE: Bootstrapping Reinforced Thinking Process to Enhance Language Model Reasoning31 Jan
  5. 2024Language Models are Hidden Reasoners: Unlocking Latent Reasoning Capabilities via Self-Rewarding6 Nov

Other references · 48

  1. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  3. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  4. 2024Qwen2.5 Technical Reportcited by 4,942
  5. 2013Auto-Encoding Variational Bayescited by 4,053
  6. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
  7. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,810
  8. 2024LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Modelscited by 1,464
  9. 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learningcited by 1,444
  10. 2025Understanding R1-Zero-Like Training: A Critical Perspectivecited by 1,345
  11. 2024OpenAI o1 System Cardcited by 1,244
  12. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMscited by 1,210
36 more
  1. 2025Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?cited by 982
  2. 2022Solving math word problems with process- and outcome-based feedbackcited by 901
  3. 2025Group Sequence Policy Optimizationcited by 835
  4. 2023RAFT: Reward rAnked FineTuning for Generative Foundation Model Alignmentcited by 686
  5. 2025SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-trainingcited by 672
  6. 2023Eureka: Human-Level Reward Design via Coding Large Language Modelscited by 571
  7. 2025Process Reinforcement through Implicit Rewardscited by 504
  8. 2025The Entropy Mechanism of Reinforcement Learning for Reasoning Language Modelscited by 485
  9. 2015Importance Weighted Autoencoderscited by 456
  10. 2025RAGEN: Understanding Self-Evolution in LLM Agents via Multi-Turn Reinforcement Learningcited by 386
  11. 2025OpenThoughts: Data Recipes for Reasoning Modelscited by 304
  12. 2025VAPO: Efficient and Reliable Reinforcement Learning for Advanced Reasoning Taskscited by 285
  13. 2025SWE-smith: Scaling Data for Software Engineering Agentscited by 285
  14. 2025Reasoning with Exploration: An Entropy Perspectivecited by 281
  15. 2025SWE-RL: Advancing LLM Reasoning via Reinforcement Learning on Open Software Evolutioncited by 263
  16. 2025Logic-RL: Unleashing LLM Reasoning with Rule-Based Reinforcement Learningcited by 262
  17. 2025REINFORCE++: Stabilizing Critic-Free Policy Optimization with Global Advantage Normalizationcited by 241
  18. 2024VinePPO: Refining Credit Assignment in RL Training of LLMscited by 183
  19. 2025RL's Razor: Why Online Reinforcement Learning Forgets Lesscited by 176
  20. 2025Light-R1: Curriculum SFT, DPO and RL for Long COT from Scratch and Beyondcited by 175
  21. 2023TACO: Topics in Algorithmic COde generation datasetcited by 160
  22. 2025SimpleTIR: End-to-End Reinforcement Learning for Multi-Turn Tool-Integrated Reasoningcited by 146
  23. 2025General-Reasoner: Advancing LLM Reasoning Across All Domainscited by 135
  24. 2025LLMs Can Easily Learn to Reason from Demonstrations Structure, not content, is what matters!cited by 132
  25. 2025On the Generalization of SFT: A Reinforcement Learning Perspective with Reward Rectificationcited by 127
  26. 2025Towards System 2 Reasoning in LLMs: Learning How to Think With Meta Chain-of-Thoughtcited by 109
  27. 2025VerlTool: Towards Holistic Agentic Reinforcement Learning with Tool Usecited by 96
  28. 2025A Sober Look at Progress in Language Model Reasoning: Pitfalls and Paths to Reproducibilitycited by 93
  29. 2024On Designing Effective RL Reward at Training Time for LLM Reasoningcited by 87
  30. 2025AceReason-Nemotron: Advancing Math and Code Reasoning through Reinforcement Learningcited by 87
  31. 2025RLPR: Extrapolating RLVR to General Domains without Verifierscited by 85
  32. 2025Reinforcing General Reasoning without Verifierscited by 77
  33. 2025CodeI/O: Condensing Reasoning Patterns via Code Input-Output Predictioncited by 55
  34. 2025Beyond Verifiable Rewards: Scaling Reinforcement Learning for Language Models to Unverifiable Datacited by 29
  35. 2025Direct Reasoning Optimization: Token-Level Reasoning Reflectivity Meets Rubric Gates for Unverifiable Taskscited by 18
  36. 2025Reinforcement Learning Teachers of Test Time Scalingcited by 10

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 7.

8 papers.

  1. 2026Privileged Information Distillation for Language Modelscited by 90
  2. 2026Learning from Language Feedback via Variational Policy Distillationcited by 7
  3. 2026Learning from Your Own Mistakes: Constructing Learnable Micro-Reflective Trajectories for Self-Distillationcited by 4
  4. 2025Coupled Variational Reinforcement Learning for Language Model General Reasoningcited by 2
  5. 2026Multimodal Continuous Reasoning via Asymmetric Mutual Variational Learningcited by 1
  6. 2026A First-Principles Theory of Slow Thinking and Active Perceptioncited by 0
  7. 2026Latent Memory Palace: Reasoning for Control as Autoregressive Variational Inferencecited by 0
  8. 2026Predictive Divergence Masks for LLM RLcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Uncovering Latent Reasoning Strategies in Language Models
  2. 2025RAVR: Reference-Answer-guided Variational Reasoning for Large Language Models
  3. 2026Efficient LLM Reasoning via Variational Posterior Guidance with Efficiency Awareness
  4. 2026From Reasoning Traces to Reusable Modules: Understanding Compositional Generalization in Language Model Reasoning
  5. 2025Sample Smart, Not Hard: Correctness-First Decoding for Better Reasoning in LLMs
  6. 2025Reasoning Under Uncertainty: Exploring Probabilistic Reasoning Capabilities of LLMs
  7. 2025A Theoretical Lens for RL-Tuned Language Models via Energy-Based Models
  8. 2022ThinkSum: Probabilistic reasoning over sets using large language models