mlsanityData to 7 Oct 2026

Multi-Agent Design: Optimizing Agents with Better Prompts and Topologies

arXiv 2502.02533 ↗First posted 4 Feb 2025cs.LG cs.AI cs.CLcited by 106

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

Large language models, employed as multiple agents that interact and collaborate with each other, have excelled at solving complex tasks. The agents are programmed with prompts that declare their functionality, along with the topologies that orchestrate interactions across agents. Designing prompts and topologies for multi-agent systems (MAS) is inherently complex. To automate the entire design process, we first conduct an in-depth analysis of the design space aiming to understand the factors behind building effective MAS. We reveal that prompts together with topologies play critical roles in enabling more effective MAS design. Based on the insights, we propose Multi-Agent System Search (MASS), a MAS optimization framework that efficiently exploits the complex MAS design space by interleaving its optimization stages, from local to global, from prompts to topologies, over three stages: 1) block-level (local) prompt optimization; 2) workflow topology optimization; 3) workflow-level (global) prompt optimization, where each stage is conditioned on the iteratively optimized prompts/topologies from former stages. We show that MASS-optimized multi-agent systems outperform a spectrum of existing alternatives by a substantial margin. Based on the MASS-found systems, we finally propose design principles behind building effective multi-agent systems.

Key facts. Cited by 106 papers in the map (data through 7 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 64 of its 86 references that are in the map. First posted on arXiv 4 Feb 2025.

Builds on

64 of 86 references.

Closest ancestors · 16

Same direction, newest first

  1. 2025Learn-by-interact: A Data-Centric Framework for Self-Adaptive Agents in Realistic Environments18 Jan
  2. 2024How to Correctly do Semantic Backpropagation on Language-based Agentic Systems4 Dec
  3. 2024AgentSquare: Automatic LLM Agent Search in Modular Design Space8 Oct
  4. 2024Cut the Crap: An Economical Communication Pipeline for LLM-based Multi-Agent Systems3 Oct
  5. 2024Automated Design of Agentic Systems15 Aug
  6. 2024Symbolic Learning Enables Self-Evolving Agents26 Jun
  7. 2024Improving Multi-Agent Debate with Sparse Communication Topology17 Jun
  8. 2024Scaling Large Language Model-based Multi-Agent Collaboration11 Jun
  9. 2024Mixture-of-Agents Enhances Large Language Model Capabilities7 Jun
  10. 2024Chain of Agents: Large Language Models Collaborating on Long-Context Tasks4 Jun
  11. 2024On the Brittle Foundations of ReAct Prompting for Agentic Large Language Models22 May
  12. 2024Rethinking the Bounds of LLM Reasoning: Are Multi-Agent Discussions the Key?28 Feb
4 more
  1. 2024Offline Training of Language Model Agents with Functions as Learnable Weights17 Feb
  2. 2024More Agents Is All You Need3 Feb
  3. 2023A Dynamic LLM-Powered Agent Network for Task-Oriented Agent Collaboration3 Oct
  4. 2023AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent Behaviors21 Aug

Other references · 48

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  3. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  4. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  5. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  6. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  7. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  8. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  9. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  10. 2021Program Synthesis with Large Language Modelscited by 3,509
  11. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
  12. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
36 more
  1. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
  2. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 1,906
  3. 2018DARTS: Differentiable Architecture Searchcited by 1,782
  4. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,739
  5. 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,378
  6. 2020Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Stepscited by 1,259
  7. 2022Large Language Models Are Human-Level Prompt Engineerscited by 1,020
  8. 2023Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debatecited by 995
  9. 2023Teaching Large Language Models to Self-Debugcited by 963
  10. 2021MuSiQue: Multihop Questions via Single-hop Question Compositioncited by 893
  11. 2019DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphscited by 883
  12. 2023Large Language Models as Optimizerscited by 830
  13. 2023Automatic Prompt Optimization with "Gradient Descent" and Beam Searchcited by 628
  14. 2024Agentless: Demystifying LLM-based Software Engineering Agentscited by 400
  15. 2024AFlow: Automating Agentic Workflow Generationcited by 395
  16. 2023ReConcile: Round-Table Conference Improves Reasoning via Consensus among Diverse LLMscited by 297
  17. 2024Optimizing Instructions and Demonstrations for Multi-Stage Language Model Programscited by 270
  18. 2024Debating with More Persuasive LLMs Leads to More Truthful Answerscited by 255
  19. 2025Agentic Retrieval-Augmented Generation: A Survey on Agentic RAGcited by 242
  20. 2023Exploring Collaboration Mechanisms for LLM Agents: A Social Psychology Viewcited by 221
  21. 2018Neural Architecture Search with Bayesian Optimisation and Optimal Transportcited by 192
  22. 2023Neural Architecture Search: Insights from 1000 Paperscited by 136
  23. 2024InfiAgent-DABench: Evaluating Agents on Data Analysis Taskscited by 134
  24. 2024DS-Agent: Automated Data Science by Empowering Large Language Models with Case-Based Reasoningcited by 119
  25. 2024Agent-Pro: Learning to Evolve via Policy-Level Reflection and Optimizationcited by 93
  26. 2023Batch Calibration: Rethinking Calibration for In-Context Learning and Prompt Engineeringcited by 91
  27. 2024AutoAct: Automatic Agent Learning from Scratch for QA via Self-Planningcited by 81
  28. 2024AutoFlow: Automated Workflow Generation for Large Language Model Agentscited by 76
  29. 2024Trace is the Next AutoDiff: Generative Optimization with Rich Feedback, Execution Traces, and LLMscited by 62
  30. 2024Archon: An Architecture Search Framework for Inference-Time Techniquescited by 59
  31. 2024Astute RAG: Overcoming Imperfect Retrieval Augmentation and Knowledge Conflicts for Large Language Modelscited by 57
  32. 2024Teach Better or Show Smarter? On Instructions and Exemplars in Automatic Prompt Optimizationcited by 28
  33. 2024Fairer Preferences Elicit Improved Human-Aligned Large Language Model Judgmentscited by 25
  34. 2022On Redundancy and Diversity in Cell-based Neural Architecture Searchcited by 18
  35. 2023Just Ask One More Time! Self-Agreement Improves Reasoning of Language Models in (Almost) All Scenarioscited by 18
  36. 2023Survival of the Most Influential Prompts: Efficient Black-Box Prompt Search via Clustering and Pruningcited by 17

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 39; 2026: 67.

106 papers.

Sort:

By area:

  1. 2025A Survey of Self-Evolving Agents: What, When, How, and Where to Evolve on the Path to Artificial Super Intelligencecited by 155
  2. 2025Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systemscited by 125
  3. 2025A Survey of Frontiers in LLM Reasoning: Inference Scaling, Learning to Reason, and Agentic Systemscited by 108
  4. 2025From Prompt Injections to Protocol Exploits: Threats in LLM-Powered AI Agents Workflowscited by 57
  5. 2025Assemble Your Crew: Automatic Multi-agent Communication Topology Design via Autoregressive Graph Generationcited by 57
  6. 2026A Survey of Agentic Reasoning for Large Language Models: Towards Recursively Self-Improving and Collective Agentscited by 47
  7. 2025A Survey on the Optimization of Large Language Model-based Agentscited by 45
  8. 2025EvoAgentX: An Automated Framework for Evolving Agentic Workflowscited by 43
  9. 2025Understanding the Information Propagation Effects of Communication Topologies in LLM-based Multi-Agent Systemscited by 39
  10. 2025EvoTest: Evolutionary Test-Time Learning for Self-Improving Agentic Systemscited by 34
  11. 2025What Prompts Don't Say: Understanding and Managing Underspecification in LLM Promptscited by 32
  12. 2025Internet of Agents: Fundamentals, Applications, and Challengescited by 31
38 more
  1. 2026MAS-Orchestra: Understanding and Improving Multi-Agent Reasoning Through Holistic Orchestration and Controlled Benchmarkscited by 26
  2. 2025Where Did It All Go Wrong? A Hierarchical Look into Multi-Agent Error Attributioncited by 25
  3. 2025Optimas: Optimizing Compound AI Systems with Globally Aligned Local Rewardscited by 23
  4. 2026From Static Templates to Dynamic Runtime Graphs: A Survey of Workflow Optimization for LLM Agentscited by 22
  5. 2025DeepWideSearch: Benchmarking Depth and Width in Agentic Information Seekingcited by 20
  6. 2025SEW: Self-Evolving Agentic Workflows for Automated Code Generationcited by 18
  7. 2026Rethinking the Value of Multi-Agent Workflow: A Strong Single Agent Baselinecited by 18
  8. 2025MAPRO: Recasting Multi-Agent Prompt Optimization as Maximum a Posteriori Inferencecited by 15
  9. 2025MermaidFlow: Redefining Agentic Workflow Generation via Safety-Constrained Evolutionary Programmingcited by 14
  10. 2025Graph-Augmented Large Language Model Agents: Current Progress and Future Prospectscited by 14
  11. 2026ReSkill: Reconciling Skill Creation with Policy Optimization in Agentic RLcited by 13
  12. 2026LatentMem: Customizing Latent Memory for Multi-Agent Systemscited by 12
  13. 2026Effective Strategies for Asynchronous Software Engineering Agentscited by 12
  14. 2025Dynamic Generation of Multi-LLM Agents Communication Topologies with Graph Diffusion Modelscited by 11
  15. 2026Recursive Multi-Agent Systemscited by 10
  16. 2025Maestro: Joint Graph & Config Optimization for Reliable AI Agentscited by 9
  17. 2025Advanced Tool Learning and Selection System (ATLASS): A Closed-Loop Framework Using LLMcited by 8
  18. 2025ProRefine: Inference-Time Prompt Refinement with Textual Feedbackcited by 8
  19. 2026SkillGraph: Self-Evolving Multi-Agent Collaboration with Multimodal Graph Topologycited by 8
  20. 2025ATLAS: Adaptive Trading with LLM AgentS Through Dynamic Prompt Optimization and Multi-Agent Coordinationcited by 7
  21. 2025BOAD: Discovering Hierarchical Software Engineering Agents via Bandit Optimizationcited by 7
  22. 2026Workflow-R1: Group Sub-sequence Policy Optimization for Multi-turn Workflow Constructioncited by 7
  23. 2026Better Harnesses, Smaller Models: Building 90% Cheaper Agents via Automated Harness Adaptationcited by 7
  24. 2026HarnessDev: Can LLMs Create and Evolve Their Own Agent Harness?cited by 7
  25. 2025Toward a Safe Internet of Agentscited by 6
  26. 2026ABSTRAL: Automatic Design of Multi-Agent Systems Through Iterative Refinement and Topology Optimizationcited by 6
  27. 2026Why Search When You Can Transfer? Amortized Agentic Workflow Design from Structural Priorscited by 6
  28. 2026Do Agent Optimizers Compound? A Continual-Learning Evaluation on Terminal-Bench 2.0cited by 6
  29. 2025Scaling Textual Gradients via Sampling-Based Momentumcited by 5
  30. 2025Agent-to-Agent Theory of Mind: Testing Interlocutor Awareness among Large Language Modelscited by 5
  31. 2025On the Importance of Task Complexity in Evaluating LLM-Based Multi-Agent Systemscited by 5
  32. 2026MAS-ProVe: Understanding the Process Verification of Multi-Agent Systemscited by 5
  33. 2026GoAgent: Group-of-Agents Communication Topology Generation for LLM-based Multi-Agent Systemscited by 5
  34. 2026Beyond Individual Intelligence: Surveying Collaboration, Failure Attribution, and Self-Evolution in LLM-based Multi-Agent Systemscited by 5
  35. 2025Compound AI Systems Optimization: A Survey of Methods, Challenges, and Future Directionscited by 4
  36. 2026EvoMAS: Evolutionary Generation of Multi-Agent Systemscited by 4
  37. 2026ValueFlow: Measuring the Propagation of Value Perturbations in Multi-Agent LLM Systemscited by 4
  38. 2026OpenSage: Self-programming Agent Generation Enginecited by 4

50 most cited of 106 All 106 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025MAS-ZERO: Designing Multi-Agent Systems with Zero Supervision
  2. 2026Learning to Evolve: A Self-Improving Framework for Multi-Agent Systems via Textual Parameter Graph Optimization
  3. 2026An Empirical Study of Multi-Agent Collaboration for Automated Research
  4. 2026AgentFactory: Towards Automated Agentic System Design and Optimization
  5. 2026MASPOB: Bandit-Based Prompt Optimization for Multi-Agent Systems with Graph Neural Networks
  6. 2025AgentSwift: Efficient LLM Agent Design via Value-guided Hierarchical Search
  7. 2025AutoMaAS: Self-Evolving Multi-Agent Architecture Search for Large Language Models
  8. 2026AgentGrad: Intervention-guided Prompt Optimization for Multi Agent Systems