mlsanityData to 7 Oct 2026

Epistemic Uncertainty for Test-Time Discovery

arXiv 2605.11328 ↗First posted 11 May 2026cs.LG cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Automated scientific discovery using large language models relies on identifying genuinely novel solutions. Standard reinforcement learning penalizes high-variance mutations, which leads the policy to prioritize familiar patterns. As a result, the maximum reward plateaus even as the average reward increases. Overcoming this limitation requires a signal that distinguishes unexplored regions from intrinsically difficult problems. This necessitates measuring disagreement across independently adapted weight hypotheses rather than relying on a single network's confidence. UG-TTT addresses this challenge by maintaining a small ensemble of low-rank adapters over a frozen base model. The per-token disagreement, quantified as the mutual information between ensemble predictions and weight hypotheses, isolates epistemic uncertainty and identifies positions where insufficient coverage leads to adapter divergence rather than intrinsic problem difficulty. This measure is incorporated as an exploration bonus into the policy gradient, directing the policy toward positions where persistent adapter disagreement signals low training coverage, the same frontier where genuine discovery is possible. A nuclear norm regularizer ensures the adapters remain distinct from one another, thereby preserving the exploration signal throughout training. Across four scientific discovery benchmarks, UG-TTT increases the maximum reward on three tasks, maintains substantially higher solution diversity, and an ablation study confirms that the regularizer is essential for sustaining this behavior.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 43 of its 49 references that are in the map. First posted on arXiv 11 May 2026.

Builds on

43 of 49 references.

References · 43

  1. 2025Qwen3 Technical Reportcited by 8,195
  2. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  3. 2019Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methodscited by 903
  4. 2020A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challengescited by 813
  5. 2025AlphaEvolve: A coding agent for scientific and algorithmic discoverycited by 763
  6. 2025Beyond the 80/20 Rule: High-Entropy Minority Tokens Drive Effective Reinforcement Learning for LLM Reasoningcited by 540
  7. 2025Reasoning with Exploration: An Entropy Perspectivecited by 277
  8. 2023Guiding Pretraining in Reinforcement Learning with Large Language Modelscited by 185
  9. 2024Teaching Large Language Models to Reason with Reinforcement Learningcited by 158
  10. 2024Scaling Laws for Reward Model Overoptimization in Direct Alignment Algorithmscited by 139
  11. 2023LM-Polygraph: Uncertainty Estimation for Language Modelscited by 130
  12. 2024To Believe or Not to Believe Your LLMcited by 120
31 more
  1. 2024A Survey on Uncertainty Quantification of Large Language Models: Taxonomy, Open Research Challenges, and Future Directionscited by 115
  2. 2024Benchmarking Uncertainty Quantification Methods for Large Language Models with LM-Polygraphcited by 105
  3. 2026Learning to Discover at Test Timecited by 94
  4. 2025Uncertainty Quantification and Confidence Calibration in Large Language Models: A Surveycited by 93
  5. 2024The Surprising Effectiveness of Test-Time Training for Few-Shot Learningcited by 88
  6. 2023Test-Time Training on Nearest Neighbors for Large Language Modelscited by 82
  7. 2023LoRA ensembles for large language model fine-tuningcited by 65
  8. 2025A Survey of Uncertainty Estimation Methods on Large Language Modelscited by 57
  9. 2023Uncertainty-Penalized Reinforcement Learning from Human Feedback with Diverse Reward LoRA Ensemblescited by 55
  10. 2025Test-Time Learning for Large Language Modelscited by 46
  11. 2023Large Language Models are Zero Shot Hypothesis Proposerscited by 45
  12. 2024Efficient Exploration for LLMscited by 44
  13. 2023Look Before You Leap: An Exploratory Study of Uncertainty Measurement for Large Language Modelscited by 40
  14. 2024Uncertainty quantification in fine-tuned LLMs using LoRA ensemblescited by 35
  15. 2020Deeply Uncertain: Comparing Methods of Uncertainty Quantification in Deep Learning Algorithmscited by 34
  16. 2025Navigate the Unknown: Enhancing LLM Reasoning with Intrinsic Motivation Guided Explorationcited by 33
  17. 2025Inference-Time Reward Hacking in Large Language Modelscited by 32
  18. 2025AutoDiscovery: Open-ended Scientific Discovery via Bayesian Surprisecited by 28
  19. 2025CDE: Curiosity-Driven Exploration for Efficient Reinforcement Learning in Large Language Modelscited by 28
  20. 2022Disentangling Epistemic and Aleatoric Uncertainty in Reinforcement Learningcited by 27
  21. 2025Enhancing Efficiency and Exploration in Reinforcement Learning for LLMscited by 23
  22. 2025Revisiting Uncertainty Estimation and Calibration of Large Language Modelscited by 22
  23. 2022R'enyi State Entropy for Exploration Acceleration in Reinforcement Learningcited by 18
  24. 2025Continuous Self-Improvement of Large Language Models by Test-time Training with Verifier-Driven Sample Selectioncited by 13
  25. 2023mLoRA: Fine-Tuning LoRA Adapters via Highly-Efficient Pipeline Parallelism in Multiple GPUscited by 12
  26. 2025Know When to Explore: Difficulty-Aware Certainty as a Guide for LLM Reinforcement Learningcited by 12
  27. 2025Large language models as uncertainty-calibrated optimizers for experimental discoverycited by 10
  28. 2025WorldLLM: Improving LLMs' world modeling using curiosity-driven theory-makingcited by 10
  29. 2025Advantage Shaping as Surrogate Reward Maximization: Unifying Pass@K Policy Gradientscited by 10
  30. 2021Bayesian Active Learning by Disagreements: A Geometric Perspectivecited by 9
  31. 2025Test-Time Training Scaling Laws for Chemical Exploration in Drug Designcited by 1

Built on by

Not cited yet.