mlsanityData to 14 Oct 2026

Seek in the Dark: Reasoning via Test-Time Instance-Level Policy Gradient in Latent Space

arXiv 2505.13308 ↗First posted 19 May 2025cs.LG cs.AI cs.CLcited by 43

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

Reasoning ability, a core component of human intelligence, continues to pose a significant challenge for Large Language Models (LLMs) in the pursuit of AGI. Although model performance has improved under the training scaling law, significant challenges remain, particularly with respect to training algorithms, such as catastrophic forgetting, and the limited availability of novel training data. As an alternative, test-time scaling enhances reasoning performance by increasing test-time computation without parameter updating. Unlike prior methods in this paradigm focused on token space, we propose leveraging latent space for more effective reasoning and better adherence to the test-time scaling law. We introduce LatentSeek, a novel framework that enhances LLM reasoning through Test-Time Instance-level Adaptation (TTIA) within the model's latent space. Specifically, LatentSeek leverages policy gradient to iteratively update latent representations, guided by self-generated reward signals. LatentSeek is evaluated on a range of reasoning benchmarks, including GSM8K, MATH-500, and AIME2024, across multiple LLM architectures. Results show that LatentSeek consistently outperforms strong baselines, such as Chain-of-Thought prompting and fine-tuning-based methods. Furthermore, our analysis demonstrates that LatentSeek is highly efficient, typically converging within a few iterations for problems of average complexity, while also benefiting from additional iterations, thereby highlighting the potential of test-time scaling in the latent space. These findings position LatentSeek as a lightweight, scalable, and effective solution for enhancing the reasoning capabilities of LLMs.

Key facts. Cited by 43 papers in the map (data through 14 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 61 of its 80 references that are in the map. First posted on arXiv 19 May 2025.

Builds on

61 of 80 references.

Closest ancestors · 9

Same direction, newest first

  1. 2025Genius: A Generalizable and Purely Unsupervised Self-Training Framework For Advanced Reasoning11 Apr
  2. 2025L1: Controlling How Long A Reasoning Model Thinks With Reinforcement Learning6 Mar
  3. 2025Wider or Deeper? Scaling LLM Inference-Time Compute with Adaptive Branching Tree Search6 Mar
  4. 2025Multi-Agent Verification: Scaling Test-Time Compute with Multiple Verifiers27 Feb
  5. 2025Can 1B LLM Surpass 405B LLM? Rethinking Compute-Optimal Test-Time Scaling10 Feb
  6. 2025Demystifying Long Chain-of-Thought Reasoning in LLMs5 Feb
  7. 2025s1: Simple test-time scaling31 Jan
  8. 2024Compressed Chain of Thought: Efficient Reasoning Through Dense Representations17 Dec
  9. 2024Language Models are Hidden Reasoners: Unlocking Latent Reasoning Capabilities via Self-Rewarding6 Nov

Other references · 52

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2023GPT-4 Technical Reportcited by 18,396
  4. 2022Training language models to follow instructions with human feedbackcited by 15,971
  5. 2017Proximal Policy Optimization Algorithmscited by 15,215
  6. 2024The Llama 3 Herd of Modelscited by 14,715
  7. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  8. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  9. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  10. 2025Qwen3 Technical Reportcited by 8,820
  11. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  12. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
38 more
  1. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  2. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  3. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  4. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
  5. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,915
  6. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,795
  7. 2023Let's Verify Step by Stepcited by 3,789
  8. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
  9. 2024Qwen2 Technical Reportcited by 2,120
  10. 2024LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Modelscited by 1,464
  11. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,407
  12. 2024Tulu 3: Pushing Frontiers in Open Language Model Post-Trainingcited by 1,053
  13. 2025Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?cited by 982
  14. 2023Reasoning with Language Model is Planning with World Modelcited by 861
  15. 2019Plug and Play Language Models: A Simple Approach to Controlled Text Generationcited by 799
  16. 2024Training Large Language Models to Reason in a Continuous Latent Spacecited by 797
  17. 2019CTRL: A Conditional Transformer Language Model for Controllable Generationcited by 762
  18. 2021GPT Understands, Toocited by 716
  19. 2021P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and Taskscited by 689
  20. 2024Self-Rewarding Language Modelscited by 681
  21. 2023An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuningcited by 639
  22. 2024Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Modelscited by 557
  23. 2025SimpleRL-Zoo: Investigating and Taming Zero Reinforcement Learning for Open Base Models in the Wildcited by 511
  24. 2022RLPrompt: Optimizing Discrete Text Prompts with Reinforcement Learningcited by 366
  25. 2024Learning to (Learn at Test Time): RNNs with Expressive Hidden Statescited by 355
  26. 2025Absolute Zero: Reinforced Self-play Reasoning with Zero Datacited by 344
  27. 2024Skywork-Reward: Bag of Tricks for Reward Modeling in LLMscited by 327
  28. 2023Statistical Rejection Sampling Improves Preference Optimizationcited by 323
  29. 2023Reward Design with Language Modelscited by 261
  30. 2024From Explicit CoT to Implicit CoT: Learning to Internalize CoT Step by Stepcited by 235
  31. 2025CODI: Compressing Chain-of-Thought into Continuous Space via Self-Distillationcited by 225
  32. 2023Chain of Hindsight Aligns Language Models with Feedbackcited by 148
  33. 2023Test-Time Training on Nearest Neighbors for Large Language Modelscited by 88
  34. 2022Recitation-Augmented Language Modelscited by 65
  35. 2025RedStar: Does Scaling Long-CoT Data Unlock Better Slow-Reasoning Systems?cited by 60
  36. 2024Self-Consistency Preference Optimizationcited by 50
  37. 2022Language Models as Inductive Reasonerscited by 32
  38. 2025Amulet: ReAlignment During Test Time for Personalized Preference Adaptation of LLMscited by 29

50 most cited of 52 All 52 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 16; 2026: 27.

43 papers.

Sort:

By area:

  1. 2025MemGen: Weaving Generative Latent Memory for Self-Evolving Agentscited by 109
  2. 2025A Survey on Latent Reasoningcited by 81
  3. 2025Reasoning Beyond Language: A Comprehensive Survey on Latent Chain-of-Thought Reasoningcited by 78
  4. 2025Beyond Pass@1: Self-Play with Variational Problem Synthesis Sustains RLVRcited by 54
  5. 2025Implicit Reasoning in Large Language Models: A Comprehensive Surveycited by 43
  6. 2026The Latent Space: Foundation, Evolution, Mechanism, Ability, and Outlookcited by 43
  7. 2025VisMem: Latent Vision Memory Unlocks Potential of Vision-Language Modelscited by 37
  8. 2025Reasoning Within the Mind: Dynamic Multimodal Interleaving in Latent Spacecited by 28
  9. 2025SwiReasoning: Switch-Thinking in Latent and Explicit for Pareto-Superior Reasoning LLMscited by 22
  10. 2026Toward Efficient Agents: Memory, Tool learning, and Planningcited by 16
  11. 2026LatentMem: Customizing Latent Memory for Multi-Agent Systemscited by 15
  12. 2026Bridging the Gap Between Latent and Explicit Reasoning with Looped Transformerscited by 13
31 more
  1. 2025Native Parallel Reasoner: Reasoning in Parallelism via Self-Distilled Reinforcement Learningcited by 12
  2. 2026MedSynapse-V: Bridging Visual Perception and Clinical Intuition via Latent Memory Evolutioncited by 12
  3. 2025Dynamic Generation of Multi-LLM Agents Communication Topologies with Graph Diffusion Modelscited by 11
  4. 2026Residual Context Diffusion Language Modelscited by 10
  5. 2025LatentEvolve: Self-Evolving Test-Time Scaling in Latent Spacecited by 9
  6. 2025MILR: Improving Multimodal Image Generation via Test-Time Latent Reasoningcited by 8
  7. 2025LaRe: Latent Refocusing for Multimodal Reasoningcited by 8
  8. 2026FlashMem: Distilling Intrinsic Latent Memory via Computation Reusecited by 8
  9. 2026Show, Don't Tell: Morphing Latent Reasoning into Image Generationcited by 8
  10. 2025Thinking on the Fly: Test-Time Reasoning Enhancement via Latent Thought Policy Optimizationcited by 7
  11. 2026Beyond tokens: a unified framework for latent communication in LLM-based multi-agent systemscited by 7
  12. 2025RuleReasoner: Reinforced Rule-based Reasoning via Domain-aware Dynamic Samplingcited by 6
  13. 2026Towards Explainable Industrial Anomaly Detection via Knowledge-Guided Latent Reasoningcited by 6
  14. 2026Unlocking the Black Box of Latent Reasoning: An Interpretability-Guided Approach to Interventioncited by 3
  15. 2026How Does Reasoning Flow? Tracing Attention-Induced Information Flow for Targeted RL in LLMscited by 3
  16. 2026Latent Visual States for Efficient Multimodal Reasoningcited by 3
  17. 2025Amortized Latent Steering: Low-Cost Alternative to Test-Time Optimizationcited by 2
  18. 2026Inference-Time Rethinking with Latent Thought Vectors for Math Reasoningcited by 2
  19. 2026Thoughts-as-Planning: Latent World Models for Chain-of-Thoughts Optimization via Reinforcement Planningcited by 1
  20. 2026Beyond the Eye: Efficient Multimodal Reasoning via Self-Regulated Implicit Visual Toolscited by 1
  21. 2026Latent Thought Credit: Multi-Answer Credit Assignment for Latent Reasoningcited by 1
  22. 2026Closing the Loop on Latent Reasoning via Test-Time Reconstructioncited by 0
  23. 2026Latent Noise Mask for Reducing Visual Redundancy in Multimodal Large Language Modelscited by 0
  24. 2026Policy Gradient Steering: Interventions from Behavioral Objectivescited by 0
  25. 2026GradCuit: Credit-Assigned Gradient Flow Enables Robust and Interpretable Test-Time Latent Reasoningcited by 0
  26. 2026Beyond Confidence: Stability-Aware Test-Time Adaptation for LLM Reasoningcited by 0
  27. 2026ThinkFlow: Self-Evolving Probabilistic Latent Memory for Lifelong Conversational Agentscited by 0
  28. 2026GenoMorph: Pathway-Grounded Genomic Disease Reasoning via Adaptive Latent Computationcited by 0
  29. 2026Token-Disentangled Latent Test-Time Scaling for Vision-Language Reasoningcited by 0
  30. 2026Hesitation Has a Geometry: Entropy-Trained Hyperbolic Probes for Sparse Activation Steeringcited by 0
  31. 2026Towards In-Parameter Memory Augmentation for Large Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Prompting Test-Time Scaling Is A Strong LLM Reasoning Data Augmentation
  2. 2025Encode, Think, Decode: Scaling test-time reasoning with recursive latent thoughts
  3. 2025Rethinking Inference-Time Scaling: Efficiency Limits and Linguistic Signals
  4. 2025Less is More: Improving LLM Reasoning with Minimal Test-Time Intervention
  5. 2026Efficient Reasoning via Constrained Optimization in Latent Space
  6. 2025Think Twice: Enhancing LLM Reasoning by Scaling Multi-round Test-time Thinking
  7. 2025Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach
  8. 2026ThinkBooster: A Unified Framework for Seamless Test-Time Scaling of LLM Reasoning