mlsanityData to 7 Oct 2026

SiriuS: Self-improving Multi-agent Systems via Bootstrapped Reasoning

arXiv 2502.04780 ↗First posted 7 Feb 2025cs.AIcited by 41

On the map: LLM reasoning → Reasoning for agents, planning and code

also in 2 more
Abstract (arXiv)

Multi-agent AI systems powered by large language models (LLMs) are increasingly applied to solve complex tasks. However, these systems often rely on fragile, manually designed prompts and heuristics, making optimization difficult. A key challenge in optimizing multi-agent systems is acquiring suitable training data for specialized agents. We introduce SiriuS, a self-improving, reasoning-driven optimization framework for multi-agent systems. Central to our approach is the construction of an experience library: a repository of high-quality reasoning trajectories. The library is built by retaining reasoning steps that lead to successful outcomes, providing a robust training set for optimizing multi-agent system. Additionally, we introduce a library augmentation procedure that refines unsuccessful trajectories, further enriching the library. SiriuS boosts performance by 2.86% to 21.88% on reasoning and biomedical QA and enhances agent negotiation in competitive settings. Our results show that SiriuS enhances multi-agent performance while generating reusable data for self-correction and self-play enhancement in the future.

Key facts. Cited by 41 papers in the map (data through 7 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 36 of its 51 references that are in the map. First posted on arXiv 7 Feb 2025.

Builds on

36 of 51 references.

Closest ancestors · 6

Same direction, newest first

  1. 2025Multiagent Finetuning: Self Improvement with Diverse Reasoning Chains10 Jan
  2. 2024Mixture-of-Agents Enhances Large Language Model Capabilities7 Jun
  3. 2024CoMM: Collaborative Multi-Agent, Multi-Reasoning-Path Prompting for Complex Problem Solving26 Apr
  4. 2024Rethinking the Bounds of LLM Reasoning: Are Multi-Agent Discussions the Key?28 Feb
  5. 2023Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate30 May
  6. 2023Reasoning with Language Model is Planning with World Model24 May

Other references · 30

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  3. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  4. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,748
  5. 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,906
  6. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,739
  7. 2019PubMedQA: A Dataset for Biomedical Research Question Answeringcited by 1,271
  8. 2022Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Taskscited by 1,057
  9. 2024Large Language Model based Multi-Agents: A Survey of Progress and Challengescited by 752
  10. 2023ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debatecited by 724
  11. 2024DeepSeek-V3 Technical Reportcited by 724
  12. 2022Large Language Models Can Self-Improvecited by 656
18 more
  1. 2024Self-Rewarding Language Modelscited by 654
  2. 2024Training Language Models to Self-Correct via Reinforcement Learningcited by 405
  3. 2018AI safety via debatecited by 384
  4. 2022Generating Sequences by Learning to Self-Correctcited by 271
  5. 2024TextGrad: Automatic "Differentiation" via Textcited by 250
  6. 2024Iterative Reasoning Preference Optimizationcited by 244
  7. 2018Decoupling Strategy and Generation in Negotiation Dialoguescited by 156
  8. 2024LLM Multi-Agent Systems: Challenges and Open Problemscited by 119
  9. 2024How Well Can LLMs Negotiate? NegotiationArena Platform and Analysiscited by 93
  10. 2024Small Language Models Need Strong Verifiers to Self-Correct Reasoningcited by 85
  11. 2024LLM2LLM: Boosting LLMs with Novel Iterative Data Enhancementcited by 69
  12. 2024LLM-based Multi-Agent Reinforcement Learning: Current and Future Directionscited by 67
  13. 2024Improving Large Language Models via Fine-grained Reinforcement Learning with Minimum Editing Constraintcited by 39
  14. 2024Large Language Models Can Self-Improve in Long-context Reasoningcited by 29
  15. 2023Teaching Language Models to Self-Improve through Interactive Demonstrationscited by 26
  16. 2024ReGenesis: LLMs can Grow into Reasoning Generalists via Self-Improvementcited by 17
  17. 2024Combating Adversarial Attacks with Multi-Agent Debatecited by 15
  18. 2024Self-Refinement of Language Models from External Proxy Metrics Feedbackcited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 19; 2026: 22.

41 papers.

Sort:

By area:

  1. 2025A Survey of Self-Evolving Agents: What, When, How, and Where to Evolve on the Path to Artificial Super Intelligencecited by 155
  2. 2025Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systemscited by 125
  3. 2025G-Memory: Tracing Hierarchical Memory for Multi-Agent Systemscited by 112
  4. 2025MAPoRL: Multi-Agent Post-Co-Training for Collaborative Large Language Models with Reinforcement Learningcited by 80
  5. 2025Latent Collaboration in Multi-Agent Systemscited by 67
  6. 2025MARFT: Multi-Agent Reinforcement Fine-Tuningcited by 51
  7. 2026A Survey of Agentic Reasoning for Large Language Models: Towards Recursively Self-Improving and Collective Agentscited by 47
  8. 2025Self-Improving LLM Agents at Test-Timecited by 30
  9. 2025CoMAS: Co-Evolving Multi-Agent Systems via Interaction Rewardscited by 27
  10. 2025MAGPIE: A benchmark for Multi-AGent contextual PrIvacy Evaluationcited by 27
  11. 2025MedAgentGym: A Scalable Agentic Training Environment for Code-Centric Reasoning in Biomedical Data Sciencecited by 22
  12. 2026Multi-Agent Teams Hold Experts Backcited by 16
29 more
  1. 2026Agent Primitives: Reusable Latent Building Blocks for Multi-Agent Systemscited by 11
  2. 2026Who&When Pro: Can LLMs Really Attribute Failures in AI Agents?cited by 11
  3. 2026Recursive Multi-Agent Systemscited by 10
  4. 2026The Vision Wormhole: Latent-Space Communication in Heterogeneous Multi-Agent Systemscited by 8
  5. 2026Learning Decentralized LLM Collaboration with Multi-Agent Actor Criticcited by 7
  6. 2026PrivAct: Internalizing Contextual Privacy Preservation via Multi-Agent Preference Trainingcited by 6
  7. 2026TodyComm: Task-Oriented Dynamic Communication for Multi-Round LLM-based Multi-Agent Systemcited by 5
  8. 2026Harnessing the Collective Intelligence of AI Agents in the Wild for New Discoveriescited by 5
  9. 2026StateBridge: Training-free Hidden-state Alignment for Latent Communication in LLM Multi-Agent Systemscited by 5
  10. 2025Compound AI Systems Optimization: A Survey of Methods, Challenges, and Future Directionscited by 4
  11. 2025SWIRL: A Staged Workflow for Interleaved Reinforcement Learning in Mobile GUI Controlcited by 4
  12. 2025Interactive Learning for LLM Reasoningcited by 4
  13. 2026LangMARL: Natural Language Multi-Agent Reinforcement Learningcited by 4
  14. 2025Agent WARPP: Workflow Adherence via Runtime Parallel Personalizationcited by 3
  15. 2025MapCoder-Lite: Distilling Multi-Agent Coding into a Single Small LLMcited by 3
  16. 2026MASPO: Joint Prompt Optimization for LLM-based Multi-Agent Systemscited by 3
  17. 2025Toward Personalizing Quantum Computing Education: An Evolutionary LLM-Powered Approachcited by 2
  18. 2025Grammar Search for Multi-Agent Systemscited by 2
  19. 2026MAGE: Multi-Agent Self-Evolution with Co-Evolutionary Knowledge Graphscited by 2
  20. 2025MAS2: Self-Generative, Self-Configuring, Self-Rectifying Multi-Agent Systemscited by 1
  21. 2026Traceable Latent Variable Discovery Based on Multi-Agent Collaborationcited by 1
  22. 2026Co-Evolution in Agentic Systems: Toward Self-Directed Evolution Beyond Human Designcited by 1
  23. 2025Reasoning With a Star: A Heliophysics Dataset and Benchmark for Agentic Scientific Reasoningcited by 0
  24. 2026Process-Supervised Multi-Agent Reinforcement Learning for Reliable Clinical Reasoningcited by 0
  25. 2026SAGA: Scene-Aware, Goal-Evolving Agents for Long-Horizon Strategy Game Planningcited by 0
  26. 2026TRACE: TRajectory Attribution for Automated Context Engineeringcited by 0
  27. 2026PropUQ-MAS: Propagation-Aware Uncertainty Quantification for LLM Multi-Agent Systemscited by 0
  28. 2026A Self-Evolving Multi-Agent Framework Defense against LLM Jailbreak Attackscited by 0
  29. 2026LatCom: Cross-Agent Latent Compression for Efficient Multi-Agent Collaborationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026MAS-Orchestra: Understanding and Improving Multi-Agent Reasoning Through Holistic Orchestration and Controlled Benchmarks
  2. 2025OPTAGENT: Optimizing Multi-Agent LLM Interactions Through Verbal Reinforcement Learning for Enhanced Reasoning
  3. 2025SE-Agent: Self-Evolution Trajectory Optimization in Multi-Step Reasoning with LLM-Based Agents
  4. 2026MARTI-MARS2: Scaling Multi-Agent Self-Search via Reinforcement Learning for Code Generation
  5. 2026Skill-MAS: Evolving Meta-Skill for Automatic Multi-Agent Systems
  6. 2026Single-Agent LLMs Outperform Multi-Agent Systems on Multi-Hop Reasoning Under Equal Thinking Token Budgets
  7. 2026DIANOIA: Diagnostic Decomposition and Joint Optimization for Multi-Agent Reasoning
  8. 2026AgentArk: Distilling Multi-Agent Intelligence into a Single LLM Agent