mlsanityData to 7 Oct 2026

INFUSER: Influence-Guided Self-Evolution Improves Reasoning

arXiv 2606.09052 ↗First posted 8 Jun 2026cs.LG cs.AI cs.CLcited by 0

On the map: LLM reasoning → Reasoning for agents, planning and code

Abstract (arXiv)

Self-evolution offers a scalable path to stronger reasoning: a pretrained language model improves itself with only minimal external supervision. Yet existing methods either depend on extensively curated or teacher-generated training data, or, when the generator runs unsupervised, reward it by a difficulty heuristic that need not improve the solver. We introduce INFUSER, an iterative co-training framework with two co-evolving roles: a Generator that drafts questions and reference golden answers from a pool of unstructured, automatically collected documents, and a Solver that improves by training on them. The solver is trained with standard correctness rewards against the generator-provided answers, while the generator is rewarded by an optimizer-aware influence score that measures whether each proposed question would actually improve the solver on the target distribution. Because this continuous, noisy influence score is poorly served by standard GRPO, we propose DuGRPO, a dual-normalized variant of GRPO, for generator training. Together, these turn the document pool into an adaptive curriculum that favors questions useful to the current solver, not just hard ones. On Qwen3-8B-Base, INFUSER outperforms strong self-evolution baselines with over 20% relative improvement on Olympiad and SuperGPQA benchmarks, and an 8B INFUSER co-evolving generator outperforms a frozen 32B thinking generator on math and coding. Ablations confirm each design choice is necessary, and two extensions, applying INFUSER to an instruction-finetuned anchor and augmenting it with rule-verifiable RLVR data, further demonstrate the flexibility and generalizability of the framework. Code is available at https://github.com/FFishy-git/INFUSER.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 42 of its 105 references that are in the map. First posted on arXiv 8 Jun 2026.

Builds on

42 of 105 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025R-Zero: Self-Evolving Reasoning LLM from Zero Data7 Aug
  2. 2025SPIRAL: Self-Play on Zero-Sum Games Incentivizes Reasoning via Multi-Agent Multi-Turn Reinforcement Learning30 Jun
  3. 2025ReMA: Learning to Meta-think for LLMs with Multi-Agent Reinforcement Learning12 Mar

Other references · 39

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  2. 2025Qwen3 Technical Reportcited by 8,195
  3. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  4. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,678
  5. 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learningcited by 1,371
  6. 2025Understanding R1-Zero-Like Training: A Critical Perspectivecited by 1,276
  7. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMscited by 1,193
  8. 2025Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?cited by 948
  9. 2025SimpleRL-Zoo: Investigating and Taming Zero Reinforcement Learning for Open Base Models in the Wildcited by 505
  10. 2025Process Reinforcement through Implicit Rewardscited by 478
  11. 2025Open-Reasoner-Zero: An Open Source Approach to Scaling Up Reinforcement Learning on the Base Modelcited by 450
  12. 2017Meta-SGD: Learning to Learn Quickly for Few-Shot Learningcited by 377
27 more
  1. 2025RAGEN: Understanding Self-Evolution in LLM Agents via Multi-Turn Reinforcement Learningcited by 357
  2. 2025Absolute Zero: Reinforced Self-play Reasoning with Zero Datacited by 323
  3. 2025VAPO: Efficient and Reliable Reinforcement Learning for Advanced Reasoning Taskscited by 281
  4. 2025Reinforcement Learning with Verifiable Rewards Implicitly Incentivizes Correct Reasoning in Base LLMscited by 276
  5. 2025Logic-RL: Unleashing LLM Reasoning with Rule-Based Reinforcement Learningcited by 260
  6. 2018Dataset Distillationcited by 224
  7. 2025ProRL: Prolonged Reinforcement Learning Expands Reasoning Boundaries in Large Language Modelscited by 187
  8. 2025A Survey of Reinforcement Learning for Large Reasoning Modelscited by 175
  9. 2025Skywork Open Reasoner 1 Technical Reportcited by 161
  10. 2025General-Reasoner: Advancing LLM Reasoning Across All Domainscited by 132
  11. 2025Self-Evolving Curriculum for LLM Reasoningcited by 107
  12. 2025Olmo 3cited by 95
  13. 2025Curriculum Reinforcement Learning from Easy to Hard Tasks Improves LLM Reasoningcited by 89
  14. 2025SPICE: Self-Play In Corpus Environments Improves Reasoningcited by 78
  15. 2025Reinforcement Learning for Reasoning in Small LLMs: What Works and What Doesn'tcited by 72
  16. 2025SuperGPQA: Scaling LLM Evaluation across 285 Graduate Disciplinescited by 67
  17. 2025DUMP: Automated Distribution-Level Curriculum Learning for RL-based LLM Post-trainingcited by 45
  18. 2025Chasing Moving Targets with Online Self-Play Reinforcement Learning for Safer Language Modelscited by 33
  19. 2025Guided Self-Evolving LLMs with Minimal Human Supervisioncited by 33
  20. 2026OPUS: Towards Efficient and Principled Data Selection in Large Language Model Pre-training in Every Iterationcited by 17
  21. 2024Efficacy of Language Model Self-Play in Non-Zero-Sum Gamescited by 14
  22. 2025Progressive Mastery: Customized Curriculum Learning with Guided Prompting for Mathematical Reasoningcited by 13
  23. 2025Data-Efficient RLVR via Off-Policy Influence Guidancecited by 12
  24. 2026GradAlign: Gradient-Aligned Data Selection for LLM Reinforcement Learningcited by 10
  25. 2026Scaling Self-Play with Self-Guidancecited by 9
  26. 2025Towards Understanding Self-play for LLM Reasoningcited by 8
  27. 2026Optimsyn: Influence-Guided Rubrics Optimization for Synthetic Data Generationcited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Questioning the Questions: Sustaining Self-Evolution in Reasoning Models
  2. 2026Ladders of Thought: A Self-Evolving Curriculum of Progressively Simplified Reasoning Traces
  3. 2025Socratic-Zero : Bootstrapping Reasoning via Data-Free Agent Co-evolution
  4. 2026Direct Self-Evolving Optimization: Evolving LLMs without Challenger Training
  5. 2026DARC: Decoupled Asymmetric Reasoning Curriculum for LLM Evolution
  6. 2026Learning What to Practice: Diagnosis-Guided Self-Evolution for Language Models
  7. 2026On the Generalization Gap in Self-Evolving Language Model Reasoning
  8. 2026Learning to Self-Evolve