mlsanityData to 14 Oct 2026

Learning to Refine Hidden States for Reliable LLM Reasoning

arXiv 2606.17524 ↗First posted 16 Jun 2026cs.LGcited by 0

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

Large language models show strong reasoning ability, but their internal reasoning process can remain unstable in complex multi-step settings, where early hidden-state errors may propagate to incorrect predictions. We propose ReLAR, a reinforcement-guided latent refinement framework that iteratively updates hidden representations before decoding. ReLAR maintains a compact latent reasoning state and uses learned depth and action controllers to adaptively determine both the number and direction of refinement steps. The controllers are trained with a policy gradient objective based on step-wise likelihood improvement, enabling efficient input-dependent reasoning without explicit chain-of-thought generation. Experiments on medical, mathematical, multi-hop reasoning, and open-ended generation benchmarks show that ReLAR improves accuracy, generation quality, and reasoning stability with substantially lower inference overhead than explicit reasoning baselines.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 39 of its 50 references that are in the map. First posted on arXiv 16 Jun 2026.

Builds on

39 of 50 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025Reasoning with Latent Thoughts: On the Power of Looped Transformers24 Feb
  2. 2025Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach7 Feb
  3. 2024Training Language Models to Self-Correct via Reinforcement Learning19 Sep
  4. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Models17 May

Other references · 35

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  4. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  5. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  6. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  7. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
  8. 2024Qwen2.5 Technical Reportcited by 4,942
  9. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  10. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
  11. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
  12. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
23 more
  1. 2023Mistral 7Bcited by 3,164
  2. 2022Locating and Editing Factual Associations in GPTcited by 2,691
  3. 2022Large Language Models Encode Clinical Knowledgecited by 1,830
  4. 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,336
  5. 2019PubMedQA: A Dataset for Biomedical Research Question Answeringcited by 1,288
  6. 2018Hierarchical Neural Story Generationcited by 1,193
  7. 2023Steering Language Models With Activation Engineeringcited by 922
  8. 2024Training Large Language Models to Reason in a Continuous Latent Spacecited by 797
  9. 2023Measuring Faithfulness in Chain-of-Thought Reasoningcited by 703
  10. 2021Editing Factual Knowledge in Language Modelscited by 553
  11. 2023The Falcon Series of Open Language Modelscited by 535
  12. 2016Adaptive Computation Time for Recurrent Neural Networkscited by 506
  13. 2021Linear Transformers Are Secretly Fast Weight Programmerscited by 502
  14. 2025MedGemma Technical Reportcited by 436
  15. 2025ReTool: Reinforcement Learning for Strategic Tool Use in LLMscited by 392
  16. 2023Faithful Chain-of-Thought Reasoningcited by 288
  17. 2025Logic-RL: Unleashing LLM Reasoning with Rule-Based Reinforcement Learningcited by 262
  18. 2023A Survey of Large Language Models for Healthcare: from Data, Technology, and Applications to Accountability and Ethicscited by 163
  19. 2024Improving Instruction-Following in Language Models through Activation Steeringcited by 160
  20. 2024Semantics-Adaptive Activation Intervention for LLMs via Dynamic Steering Vectorscited by 63
  21. 2024Med42 -- Evaluating Fine-Tuning Strategies for Medical LLMs: Full-Parameter vs. Parameter-Efficient Approachescited by 52
  22. 2026Verifiable Process Rewards for Agentic Reasoningcited by 11
  23. 2025ActivationReasoning: Logical Reasoning in Latent Activation Spacescited by 5

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Re2: Unlocking LLM Reasoning via Reinforcement Learning with Re-solving
  2. 2025Efficient Post-Training Refinement of Latent Reasoning in Large Language Models
  3. 2026ReTreVal: Reasoning Tree with Validation and Cross-Problem Memory for Large Language Models
  4. 2026One Refiner to Unlock Them All: Inference-Time Reasoning Elicitation via Reinforcement Query Refinement
  5. 2026Efficient LLM Reasoning via Variational Posterior Guidance with Efficiency Awareness
  6. 2025ReJump: A Tree-Jump Representation for Analyzing and Improving LLM Reasoning
  7. 2025Learning When to Stop: Adaptive Latent Reasoning via Reinforcement Learning
  8. 2025Temporal Sampling for Forgotten Reasoning in LLMs