mlsanityData to 14 Oct 2026

Self-Challenging Language Model Agents

arXiv 2506.01716 ↗First posted 2 Jun 2025cs.AI cs.CLcited by 59

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

Large language models are quickly becoming the foundation for intelligent agents that are capable of using tools. However, training such agents is challenging because it requires human creation and annotation of a diverse set of tasks, tools, and evaluation criteria. In this paper, we propose the Self-Challenging framework for training an agent on high-quality tasks that are generated by itself. The agent first plays the role of challenger and generates a task after interacting with the given tools. The tasks take the form of a novel general class of problems termed Code-as-Task, which are defined by an instruction, a verification function and solution and failure cases which serve as tests, allowing to filter only for high-quality tasks. The agent then takes an executor role and trains on those tasks with reinforcement learning using the evaluation feedback as a reward. Evaluation on two existing multi-turn tool-use agent benchmarks, M3ToolEval and TauBench, shows the Self-Challenging framework achieves over a two-fold improvement in Llama-3.1-8B-Instruct, despite using only self-generated training data.

Key facts. Cited by 59 papers in the map (data through 14 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 43 of its 50 references that are in the map. First posted on arXiv 2 Jun 2025.

Builds on

43 of 50 references.

Closest ancestors · 8

Same direction, newest first

  1. 2025Learn-by-interact: A Data-Centric Framework for Self-Adaptive Agents in Realistic Environments18 Jan
  2. 2024TheAgentCompany: Benchmarking LLM Agents on Consequential Real World Tasks18 Dec
  3. 2024Building Math Agents with Multi-Turn Iterative Preference Learning4 Sep
  4. 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains17 Jun
  5. 2024BAGEL: Bootstrapping Agents by Guiding Exploration with Language12 Mar
  6. 2024TravelPlanner: A Benchmark for Real-World Planning with Language Agents2 Feb
  7. 2024If LLM Is the Wizard, Then Code Is the Wand: A Survey on How Code Empowers Large Language Models to Serve as Intelligent Agents1 Jan
  8. 2023Augmenting Autotelic Agents with Large Language Models21 May

Other references · 35

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  3. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  4. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  5. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,373
  6. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,724
  7. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,947
  8. 2022Code as Policies: Language Model Programs for Embodied Controlcited by 1,210
  9. 2022Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agentscited by 1,035
  10. 2023VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Modelscited by 750
  11. 2024Self-Rewarding Language Modelscited by 681
  12. 2024Executable Code Actions Elicit Better LLM Agentscited by 564
23 more
  1. 2024Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Modelscited by 557
  2. 2024WebVoyager: Building an End-to-End Web Agent with Large Multimodal Modelscited by 438
  3. 2024AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agentscited by 401
  4. 2025Absolute Zero: Reinforced Self-play Reasoning with Zero Datacited by 344
  5. 2024Training Software Engineering Agents and Verifiers with SWE-Gymcited by 336
  6. 2024WebRL: Training LLM Web Agents via Self-Evolving Online Curriculum Reinforcement Learningcited by 256
  7. 2024ArCHer: Training Language Model Agents via Hierarchical Multi-Turn RLcited by 228
  8. 2024DigiRL: Training In-The-Wild Device-Control Agents with Autonomous Reinforcement Learningcited by 214
  9. 2017Intrinsic Motivation and Automatic Curricula via Asymmetric Self-Playcited by 141
  10. 2024StableToolBench: Towards Stable Large-Scale Benchmarking on Tool Learning of Large Language Modelscited by 132
  11. 2025SWEET-RL: Training Multi-Turn LLM Agents on Collaborative Reasoning Taskscited by 118
  12. 2025STP: Self-play LLM Theorem Provers with Iterative Conjecturing and Provingcited by 79
  13. 2024Beyond Browsing: API-Based Web Agentscited by 78
  14. 2024OMNI-EPIC: Open-endedness via Models of human Notions of Interestingness with Environments Programmed in Codecited by 74
  15. 2021Asymmetric self-play for automatic goal discovery in robotic manipulationcited by 72
  16. 2024DistRL: An Asynchronous Distributed Reinforcement Learning Framework for On-Device Control Agentscited by 68
  17. 2024NNetNav: Unsupervised Learning of Browser Agents Through Environment Interaction in the Wildcited by 59
  18. 2013Active Learning of Inverse Models with Intrinsically Motivated Goal Exploration in Robotscited by 58
  19. 2024Tool Learning in the Wild: Empowering Language Models as Automatic Tool Agentscited by 56
  20. 2024Proposer-Agent-Evaluator(PAE): Autonomous Skill Discovery For Foundation Model Internet Agentscited by 50
  21. 2024OpenWebVoyager: Building Multimodal Web Agents via Iterative Real-World Exploration, Feedback and Optimizationcited by 46
  22. 2024TOOLVERIFIER: Generalization to New Tools via Self-Verificationcited by 24
  23. 2025Digi-Q: Learning Q-Value Functions for Training Device-Control Agentscited by 20

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 21; 2026: 38.

59 papers.

Sort:

By area:

  1. 2025R-Zero: Self-Evolving Reasoning LLM from Zero Datacited by 228
  2. 2025A Survey of Self-Evolving Agents: What, When, How, and Where to Evolve on the Path to Artificial Super Intelligencecited by 170
  3. 2025Agent Learning via Early Experiencecited by 85
  4. 2025Agent0: Unleashing Self-Evolving Agents from Zero Data via Tool-Integrated Reasoningcited by 75
  5. 2025Your Agent May Misevolve: Emergent Risks in Self-evolving LLM Agentscited by 62
  6. 2025Multi-Agent Evolve: LLM Self-Improve through Co-evolutioncited by 60
  7. 2026A Survey of Agentic Reasoning for Large Language Models: Towards Recursively Self-Improving and Collective Agentscited by 50
  8. 2025Scaling Agent Learning via Experience Synthesiscited by 44
  9. 2026Tool-R0: Self-Evolving LLM Agents for Tool-Learning from Zero Datacited by 34
  10. 2025CoT-Self-Instruct: Building high-quality synthetic prompts for reasoning and non-reasoning taskscited by 32
  11. 2025Self-Improving LLM Agents at Test-Timecited by 30
  12. 2025A Survey on Autonomy-Induced Security Risks in Large Model-Based Agentscited by 29
38 more
  1. 2025SPELL: Self-Play Reinforcement Learning for Evolving Long-Context Language Modelscited by 29
  2. 2025Aegis: Automated Error Generation and Attribution for Multi-Agent Systemscited by 22
  3. 2025Adaptation of Agentic AI: A Survey of Post-Training, Memory, and Skillscited by 22
  4. 2026Inference-Time Scaling of Verification: Self-Evolving Deep Research Agents via Test-Time Rubric-Guided Verificationcited by 22
  5. 2026Recursive Synthesis for Long-Horizon Terminal Taskscited by 19
  6. 2025Toward Effective Tool-Integrated Reasoning via Self-Evolved Preference Learningcited by 17
  7. 2026G-Zero: Self-Play for Open-Ended Generation from Zero Datacited by 17
  8. 2026Autodata: An agentic data scientist to create high quality synthetic datacited by 16
  9. 2026SPADE: Self-Play in Adaptive Synthetic Executable Environmentscited by 12
  10. 2026Safe and Scalable Web Agent Learning via Recreated Websitescited by 11
  11. 2025MLE-Smith: Scaling MLE Tasks with Automated Multi-Agent Pipelinecited by 9
  12. 2025SusBench: An Online Benchmark for Evaluating Dark Pattern Susceptibility of Computer-Use Agentscited by 8
  13. 2025AI & Human Co-Improvement for Safer Co-Superintelligencecited by 8
  14. 2026Agentic Environment Engineering for Large Language Models: A Survey of Environment Modeling, Synthesis, Evaluation, and Applicationcited by 8
  15. 2025The Path of Self-Evolving Large Language Models: Achieving Data-Efficient Learning via Intrinsic Feedbackcited by 6
  16. 2026SEARL: Joint Optimization of Policy and Tool Graph Memory for Self-Evolving Agentscited by 6
  17. 2025Agentic Systems in Radiology: Design, Applications, Evaluation, and Challengescited by 5
  18. 2026Yunjue Agent Tech Report: A Fully Reproducible, Zero-Start In-Situ Self-Evolving Agent System for Open-Ended Taskscited by 5
  19. 2026ReSyn: Autonomously Scaling Synthetic Environments for Reasoning Modelscited by 5
  20. 2026Bootstrapping Post-training Signals for Open-ended Tasks via Rubric-based Self-play on Pre-training Textcited by 5
  21. 2026From Trainee to Trainer: LLM-Designed Training Environment for RL with Multi-Agent Reasoningcited by 5
  22. 2026Scaling Large Reasoning Models beyond Human Supervision: A Path toward Superintelligencecited by 5
  23. 2026ET-Agent: Incentivizing Effective Tool-Integrated Reasoning Agent via Behavior Calibrationcited by 4
  24. 2026BenchEvolver: Frontier Task Synthesis via Solution-Centric Evolutioncited by 4
  25. 2026On the Fragility of Self-Improving Agents: Variance, Task Order, and Underspecificationcited by 4
  26. 2025EvoCurr: Self-evolving Curriculum with Behavior Code Generation for Complex Decision-makingcited by 3
  27. 2026A Deep Dive into Scaling RL for Code Generation with Synthetic Data and Curriculacited by 3
  28. 2026Training LLM Agents for Spontaneous, Reward-Free Self-Evolution via World Knowledge Explorationcited by 3
  29. 2026PREPING: Zero-Task Agent Memory Construction via Self-Practicecited by 3
  30. 2026Internalizing the Future: A Unified Agentic Training Paradigm for World Model Planningcited by 3
  31. 2026Bifrost: Steering Strategic Trajectories to Bridge Contextual Gaps for Self-Improving Agentscited by 2
  32. 2026TTSR: Test-Time Self-Evolving via Reflectioncited by 2
  33. 2026Self-Evolving Coding Agentscited by 2
  34. 2026ReIn: Conversational Error Recovery with Reasoning Inceptioncited by 1
  35. 2026Help Without Being Asked: A Deployed Proactive Agent System for On-Call Support with Continuous Self-Improvementcited by 1
  36. 2026Learning What to Remember and What to Internalize in LLM Self-Evolution via Adaptive Memory-Parameter Coordinationcited by 1
  37. 2026Self-Play Pretraining with Zero Datacited by 1
  38. 2025Position: Vibe Coding Needs Vibe Reasoning: Improving Vibe Coding with Formal Verificationcited by 0

50 most cited of 59 All 59 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026ClawLoop: Self-Improving Agents via Harness-Native Training Loop
  2. 2025Evolving in Tasks: Empowering the Multi-modality Large Language Model as the Computer Use Agent
  3. 2026Towards AGI A Pragmatic Approach Towards Self Evolving Agent
  4. 2026SkillAdaptor: Self-Adapting Skills for LLM Agents from Trajectories
  5. 2025A Self-Improving Coding Agent
  6. 2024A Survey on Self-Evolution of Large Language Models
  7. 2025Agent-R: Training Language Model Agents to Reflect via Iterative Self-Training
  8. 2026Agent Skills for Large Language Models: Architecture, Acquisition, Security, and the Path Forward