mlsanityData to 7 Oct 2026

Language Models are Hidden Reasoners: Unlocking Latent Reasoning Capabilities via Self-Rewarding

arXiv 2411.04282 ↗First posted 6 Nov 2024cs.AI cs.CL cs.LGcited by 53

On the map: LLM reasoning → Test-time compute and efficient thinking

also in 1 more
Abstract (arXiv)

Large language models (LLMs) have shown impressive capabilities, but still struggle with complex reasoning tasks requiring multiple steps. While prompt-based methods like Chain-of-Thought (CoT) can improve LLM reasoning at inference time, optimizing reasoning capabilities during training remains challenging. We introduce LaTent Reasoning Optimization (LaTRO), a principled framework that formulates reasoning as sampling from a latent distribution and optimizes it via variational approaches. LaTRO enables LLMs to concurrently improve both their reasoning process and ability to evaluate reasoning quality, without requiring external feedback or reward models. We validate LaTRO through experiments on GSM8K and ARC-Challenge datasets using multiple model architectures. On GSM8K, LaTRO improves zero-shot accuracy by an average of 12.5% over base models and 9.6% over supervised fine-tuning across Phi-3.5-mini, Mistral-7B, and Llama-3.1-8B. Our findings suggest that pre-trained LLMs possess latent reasoning capabilities that can be unlocked and enhanced through our proposed optimization approach in a self-improvement manner. The code of LaTRO is available at urlhttps://github.com/SalesforceAIResearch/LaTRO.

Key facts. Cited by 53 papers in the map (data through 7 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 28 of its 56 references that are in the map. First posted on arXiv 6 Nov 2024.

Builds on

28 of 56 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024Training Language Models to Self-Correct via Reinforcement Learning19 Sep
  2. 2024Inference Scaling Laws: An Empirical Analysis of Compute-Optimal Inference for Problem-Solving with Language Models1 Aug
  3. 2023Large Language Models Cannot Self-Correct Reasoning Yet3 Oct

Other references · 25

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  3. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  4. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  5. 2013Auto-Encoding Variational Bayescited by 3,980
  6. 2023Let's Verify Step by Stepcited by 3,598
  7. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
  8. 2023Mistral 7Bcited by 3,065
  9. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,748
  10. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,883
  11. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
  12. 2024DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligencecited by 1,154
13 more
  1. 2024Large Language Monkeys: Scaling Inference Compute with Repeated Samplingcited by 946
  2. 2024Improve Mathematical Reasoning in Language Models by Automated Process Supervisioncited by 426
  3. 2020Generative Language Modeling for Automated Theorem Provingcited by 379
  4. 2022Self-critiquing models for assisting human evaluatorscited by 372
  5. 2018Maximum a Posteriori Policy Optimisationcited by 318
  6. 2024Quiet-STaR: Language Models Can Teach Themselves to Think Before Speakingcited by 292
  7. 2024To CoT or not to CoT? Chain-of-thought helps mainly on math and symbolic reasoningcited by 257
  8. 2024Chain-of-Thought Reasoning Without Promptingcited by 208
  9. 2024Teaching Large Language Models to Reason with Reinforcement Learningcited by 158
  10. 2023Retroformer: Retrospective Large Language Agents with Policy Gradient Optimizationcited by 110
  11. 2024Back to Basics: Revisiting REINFORCE Style Optimization for Learning from Human Feedback in LLMscited by 103
  12. 2023Amortizing intractable inference in large language modelscited by 94
  13. 2024Diversity Empowers Intelligence: Integrating Expertise of Software Engineering Agentscited by 35

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 34; 2026: 18.

53 papers.

Sort:

By area:

  1. 2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Modelscited by 381
  2. 2024Token-Budget-Aware LLM Reasoningcited by 242
  3. 2025Reinforcing General Reasoning without Verifierscited by 76
  4. 2025STAIR: Improving Safety Alignment with Introspective Reasoningcited by 72
  5. 2025Chain-of-Visual-Thought: Teaching VLMs to See and Think Better with Continuous Visual Tokenscited by 65
  6. 2025Think Before Recommend: Unleashing the Latent Reasoning Power for Sequential Recommendationcited by 50
  7. 2025Reasoning to Learn from Latent Thoughtscited by 43
  8. 2025Implicit Reasoning in Large Language Models: A Comprehensive Surveycited by 42
  9. 2025Seek in the Dark: Reasoning via Test-Time Instance-Level Policy Gradient in Latent Spacecited by 40
  10. 2025Learning to Reason for Long-Form Story Generationcited by 34
  11. 2025MoDoMoDo: Multi-Domain Data Mixtures for Multimodal LLM Reinforcement Learningcited by 29
  12. 2026Recursive Self-Improvement in AI: From Bounded Self-Refinement to Autonomous Research Loopscited by 25
38 more
  1. 2025Brains vs. Bytes: Evaluating LLM Proficiency in Olympiad Mathematicscited by 22
  2. 2025Activation Steering for Chain-of-Thought Compressioncited by 22
  3. 2025Direct Reasoning Optimization: Token-Level Reasoning Reflectivity Meets Rubric Gates for Unverifiable Taskscited by 16
  4. 2025Fast on the Easy, Deep on the Hard: Efficient Reasoning via Powered Length Penaltycited by 15
  5. 2025Iterative Self-Incentivization Empowers Large Language Models as Agentic Searcherscited by 11
  6. 2025BRiTE: Bootstrapping Reinforced Thinking Process to Enhance Language Model Reasoningcited by 10
  7. 2026VI-CuRL: Stabilizing Verifier-Independent RL Reasoning via Confidence-Guided Variance Reductioncited by 9
  8. 2025Behavior Injection: Preparing Language Models for Reinforcement Learningcited by 8
  9. 2025Variational Reasoning for Language Modelscited by 8
  10. 2025Webscale-RL: Automated Data Pipeline for Scaling RL Data to Pretraining Levelscited by 8
  11. 2025Efficient Post-Training Refinement of Latent Reasoning in Large Language Modelscited by 7
  12. 2025From Trial to Triumph: Advancing Long Video Understanding via Visual Context Sample Scaling and Self-reward Alignmentcited by 6
  13. 2025CogDual: Enhancing Dual Cognition of LLMs via Reinforcement Learning with Implicit Rule-Based Rewardscited by 6
  14. 2025SSR: Socratic Self-Refine for Large Language Model Reasoningcited by 6
  15. 2026Silence the Judge: Reinforcement Learning with Self-Verifier via Latent Geometric Clusteringcited by 6
  16. 2026LatentChem: From Textual CoT to Latent Thinking in Chemical Reasoningcited by 6
  17. 2025MetaSC: Test-Time Safety Specification Optimization for Language Modelscited by 4
  18. 2025RefGrader: Automated Grading of Mathematical Competition Proofs using Agentic Workflowscited by 3
  19. 2026Discovering Process-Outcome Credit in Multi-Step LLM Reasoningcited by 3
  20. 2026Capabilities and Fundamental Limits of Latent Chain-of-Thoughtcited by 3
  21. 2026Self-Supervised Bootstrapping of Action-Predictive Embodied Reasoningcited by 3
  22. 2026When are likely answers right? On Sequence Probability and Correctness in LLMscited by 3
  23. 2025Shaping Explanations: Semantic Reward Modeling with Encoder-Only Transformers for GRPOcited by 2
  24. 2025From Long to Lean: Performance-aware and Adaptive Chain-of-Thought Compression via Multi-round Refinementcited by 2
  25. 2025AdaThink-Med: Optimizing Inference-Time Compute for Medical Reasoning via Uncertainty Quantificationcited by 2
  26. 2025RLoop: An Self-Improving Framework for Reinforcement Learning with Iterative Policy Initializationcited by 2
  27. 2025In-Token Rationality Optimization: Towards Accurate and Concise LLM Reasoning via Self-Feedbackcited by 2
  28. 2026From Latent Signals to Reflection Behavior: Tracing Meta-Cognitive Activation Trajectory in R1-Style LLMscited by 2
  29. 2026Native Reasoning Models: Training Language Models to Reason on Unverifiable Datacited by 2
  30. 2026Scaling Large Reasoning Models beyond Human Supervision: A Path toward Superintelligencecited by 2
  31. 2025Dist2ill: Distributional Distillation for One-Pass Uncertainty Estimation in Large Language Modelscited by 1
  32. 2025Coupled Variational Reinforcement Learning for Language Model General Reasoningcited by 1
  33. 2026DARL: Encouraging Diverse Answers for General Reasoning without Verifierscited by 1
  34. 2026Why Self-Rewarding Works: Theoretical Guarantees for Iterative Alignment of Language Modelscited by 1
  35. 2026LASAR: Latent Adaptive Semantic Aligned Reasoning for Generative Recommendationcited by 1
  36. 2026Soft-SVeRL: Self-Verified Reinforcement Learning with Soft Rewardscited by 1
  37. 2025Latent Principle Discovery for Language Model Self-Improvementcited by 0
  38. 2025Learning to Reason in LLMs by Expectation Maximizationcited by 0

50 most cited of 53 All 53 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Thinking on the Fly: Test-Time Reasoning Enhancement via Latent Thought Policy Optimization
  2. 2025Beyond Chains of Thought: Benchmarking Latent-Space Reasoning Abilities in Large Language Models
  3. 2026Latent Thought Flow: Efficient Latent Reasoning in Large Language Models
  4. 2025A Survey on Latent Reasoning
  5. 2024Inference Scaling vs Reasoning: An Empirical Analysis of Compute-Optimal LLM Problem-Solving
  6. 2025Advancing Reasoning in Large Language Models: Promising Methods and Approaches
  7. 2025Encode, Think, Decode: Scaling test-time reasoning with recursive latent thoughts
  8. 2026Improving reasoning at inference time via uncertainty minimisation