mlsanityData to 14 Oct 2026

HAMMER: Hamiltonian Curiosity Augmented Large Language Model Reinforcement

arXiv 2509.25240 ↗First posted 25 Sep 2025cs.LG cs.AI cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Recent curriculum reinforcement learning for large language models (LLMs) typically rely on difficulty-based annotations for data filtering and ordering. However, such methods suffer from local optimization, where continual training on simple samples in the early steps can cause the policy to lose its exploration. We propose a novel schema, namely Hamiltonian curiosity augmented large language model reinforcement (HAMMER), that transfers diversity metrics, commonly used in dataset evaluation, into the dynamic reinforcement learning procedure, where training samples are ordered via a minimum-semantic Hamiltonian path making the initial training retrain more exploration. From a theoretical perspective of generalization bounds, diversity-driven ordering facilitates stable convergence. Empirical evaluations indicate that HAMMER stimulates model "curiosity" and consistently achieves a 3% to 4% average accuracy gain across diverse inference benchmark.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 31 of its 46 references that are in the map. First posted on arXiv 25 Sep 2025.

Builds on

31 of 46 references.

References · 31

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  3. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  4. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,724
  5. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 2,057
  6. 2017Understanding Black-box Predictions via Influence Functionscited by 1,741
  7. 2022Beyond neural scaling laws: beating power law scaling via data pruningcited by 528
  8. 2025SimpleRL-Zoo: Investigating and Taming Zero Reinforcement Learning for Open Base Models in the Wildcited by 511
  9. 2021MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence Frontierscited by 470
  10. 2024LLM2Vec: Large Language Models Are Secretly Powerful Text Encoderscited by 404
  11. 2020Curriculum Learning for Reinforcement Learning Domains: A Framework and Surveycited by 373
  12. 2022The Vendi Score: A Diversity Evaluation Metric for Machine Learningcited by 311
19 more
  1. 2025Learning to Reason under Off-Policy Guidancecited by 281
  2. 2025Logic-RL: Unleashing LLM Reasoning with Rule-Based Reinforcement Learningcited by 262
  3. 2024ReFT: Reasoning with Reinforced Fine-Tuningcited by 239
  4. 2017GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibriumcited by 213
  5. 2023Does Writing with Language Models Reduce Content Diversity?cited by 192
  6. 2025Pass@k Training for Adaptively Balancing Exploration and Exploitation of Large Reasoning Modelscited by 138
  7. 2021Scaling Up Influence Functionscited by 132
  8. 2025Online Difficulty Filtering for Reasoning Oriented Reinforcement Learningcited by 104
  9. 2025Curriculum Reinforcement Learning from Easy to Hard Tasks Improves LLM Reasoningcited by 92
  10. 2025An Empirical Study on Eliciting and Improving R1-like Reasoning Modelscited by 78
  11. 2024Scaling Data Diversity for Fine-Tuning Language Models in Human Alignmentcited by 26
  12. 2024Towards a Scalable Reference-Free Evaluation of Generative Modelscited by 26
  13. 2020DQI: Measuring Data Quality in NLPcited by 24
  14. 2023Beyond Scale: The Diversity Coefficient as a Data Quality Metric for Variability in Natural Language Datacited by 19
  15. 2024OpenRFT: Adapting Reasoning Foundation Model for Domain-specific Tasks with Reinforcement Fine-Tuningcited by 13
  16. 2024Speculative Coreset Selection for Task-Specific Fine-tuningcited by 10
  17. 2025Measuring Diversity in Synthetic Datasetscited by 5
  18. 2025Coreset Selection via LLM-based Concept Bottleneckscited by 2
  19. 2025Improving Reinforcement Learning Sample-Efficiency using Local Approximationcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026A Survey of Reinforcement Learning for Large Language Models under Data Scarcity: Challenges and Solutions
  2. 2026Exploration-Driven Optimization for Test-Time Large Language Model Reasoning
  3. 2024Entropy-Regularized Token-Level Policy Optimization for Language Agent Reinforcement
  4. 2025Representation-Based Exploration for Language Models: From Test-Time to Post-Training
  5. 2024Survey on Large Language Model-Enhanced Reinforcement Learning: Concept, Taxonomy, and Methods
  6. 2025CDE: Curiosity-Driven Exploration for Efficient Reinforcement Learning in Large Language Models
  7. 2025Rethinking Large Language Model Distillation: A Constrained Markov Decision Process Perspective
  8. 2025Curiosity-Driven Reinforcement Learning from Human Feedback