mlsanityData to 14 Oct 2026

CURIO: Curiosity-Driven Test-Time Learning for Open-Ended Discovery

arXiv 2610.04851 ↗First posted 4 Oct 2026cs.LG cs.CLcited by 0

On the map: Deep reinforcement learning methods → RL training of language and agent models

Abstract (arXiv)

Open-ended discovery requires learning from repeated attempts while continuing to explore directions whose value is not yet apparent. Search with a frozen large language model (LLM) can reuse previous solutions in context, but cannot update the model from its successes and failures on the test problem. Reinforcement learning (RL) enables such adaptation; however, strongly favoring high-reward trajectories may suppress low-reward yet potentially promising directions too early. We introduce CURIO, a curiosity-driven test-time learning framework that complements task feedback with an Intrinsic Curiosity World Model (ICWM). The ICWM learns transitions in the policy's hidden-state representation and supplies prediction-error bonuses at sampled tokens outside the policy's top-k choices. Epoch normalization and an annealed weight regulate their contribution to the policy update. On six mathematical discovery tasks and single-cell denoising with Qwen3 backbones from 8B to 235B, three-run means improve over a matched task-only RL control on five mathematical objectives, match the best reported performance on Circle Packing, and improve denoising Score and mean squared error (MSE) on both held-out corpora at every tested scale. Relative gains reach 18.3% on Hadamard and 10.8% on denoising Score. Code-diversity measurements show greater structural variation among generated programs, supporting curiosity as a complementary exploration signal for learning in open-ended discovery.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL training of language and agent models, one of 15 directions in Deep reinforcement learning methods. Builds on 28 of its 44 references that are in the map. First posted on arXiv 4 Oct 2026.

Builds on

28 of 44 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025Curiosity-Driven Reinforcement Learning from Human Feedback20 Jan

Other references · 27

  1. 2025Qwen3 Technical Reportcited by 8,820
  2. 2015Continuous control with deep reinforcement learningcited by 3,407
  3. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,194
  4. 2018Exploration by Random Network Distillationcited by 993
  5. 2025AlphaEvolve: A coding agent for scientific and algorithmic discoverycited by 867
  6. 2023Eureka: Human-Level Reward Design via Coding Large Language Modelscited by 571
  7. 2019Provably Efficient Reinforcement Learning with Linear Function Approximationcited by 520
  8. 2025The Entropy Mechanism of Reinforcement Learning for Reasoning Language Modelscited by 485
  9. 2024Automated Design of Agentic Systemscited by 407
  10. 2025KernelBench: Can LLMs Write Efficient GPU Kernels?cited by 225
  11. 2025ShinkaEvolve: Towards Open-Ended And Sample-Efficient Program Evolutioncited by 174
  12. 2022Evolution through Large Modelscited by 134
15 more
  1. 2024LLM-SR: Scientific Equation Discovery via Programming with Large Language Modelscited by 133
  2. 2023Language Model Crossover: Variation through Few-Shot Promptingcited by 122
  3. 2026Learning to Discover at Test Timecited by 115
  4. 2021A Survey of Exploration Methods in Reinforcement Learningcited by 73
  5. 2025ThetaEvolve: Test-time Learning on Open Problemscited by 64
  6. 2026PostTrainBench: Can LLM Agents Automate LLM Post-Training?cited by 64
  7. 2025CodeEvolve: an open source evolutionary coding agent for algorithmic discovery and optimizationcited by 44
  8. 2025On Entropy Control in LLM-RL Algorithmscited by 40
  9. 2025Navigate the Unknown: Enhancing LLM Reasoning with Intrinsic Motivation Guided Explorationcited by 34
  10. 2022Guarantees for Epsilon-Greedy Reinforcement Learning with Function Approximationcited by 31
  11. 2024Flow of Reasoning: Training LLMs for Divergent Reasoning with Minimal Examplescited by 30
  12. 2025Online Preference Alignment for Language Models via Count-based Explorationcited by 26
  13. 2026Structured Scaling of AI Discovery Across Diverse Scientific Domainscited by 21
  14. 2026NatureBench: Can Coding Agents Match the Published SOTA of Nature-Family Papers?cited by 10
  15. 2026Weak-to-Strong On-Policy Distillationcited by 8

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025CDE: Curiosity-Driven Exploration for Efficient Reinforcement Learning in Large Language Models
  2. 2026Endogenous Exploration in Reinforcement Learning with Intrinsic Curiosity
  3. 2026Curiosity-Critic: Cumulative Prediction Error Improvement as a Tractable Intrinsic Reward for World Model Training
  4. 2025Representation-Based Exploration for Language Models: From Test-Time to Post-Training
  5. 2026Can In-Context Learning Support Intrinsic Curiosity?
  6. 2026PACEvolve++: Improving Test-time Learning for Evolutionary Search Agents
  7. 2026Look Inward to Explore Outward: Learning Temperature Policy from LLM Internal States via Hierarchical RL
  8. 2025In-Context Curiosity: Distilling Exploration for Decision-Pretrained Transformers on Bandit Tasks