mlsanityData to 14 Oct 2026

Optimizing Sequential Multi-Step Tasks with Parallel LLM Agents

arXiv 2507.08944 ↗First posted 11 Jul 2025cs.MA cs.AIcited by 14

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

Large language model (LLM)-based multi-agent systems have demonstrated remarkable promise for tackling complex tasks by breaking them down into subtasks that are iteratively planned, executed, observed, and refined. Despite their effectiveness, these systems often incur high latency because real-world problems frequently demand multiple iterative cycles of reasoning steps. To address this challenge, we propose M1-Parallel, a framework that concurrently runs multiple multi-agent teams in parallel to uncover distinct solution paths. By leveraging an event-driven communication model with asynchronous messaging, M1-Parallel efficiently capitalizes on the inherent diversity of valid plans to either reduce end-to-end latency or boost task completion rates. Our experiments on complex tasks show that M1-Parallel with early termination achieves up to 2.2× speedup while preserving accuracy, and that M1-Parallel with aggregation yields higher task completion rates. We further investigate strategies aimed at encouraging diverse execution plans but observe no additional performance gains over repeated sampling. Overall, these findings underscore the potential of parallel plan execution for optimizing multi-agent systems for real-world, high-complexity reasoning tasks.

Key facts. Cited by 14 papers in the map (data through 14 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 35 of its 39 references that are in the map. First posted on arXiv 11 Jul 2025.

Builds on

35 of 39 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024Mixture-of-Agents Enhances Large Language Model Capabilities7 Jun
  2. 2024Large Language Model based Multi-Agents: A Survey of Progress and Challenges21 Jan
  3. 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework1 Aug
  4. 2023Unleashing the Emergent Cognitive Synergy in Large Language Models: A Task-Solving Agent through Multi-Persona Self-Collaboration11 Jul

Other references · 31

  1. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  2. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  3. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  4. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  5. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
  6. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
  7. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,073
  8. 2024Mixtral of Expertscited by 1,823
  9. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,784
  10. 2022Fast Inference from Transformers via Speculative Decodingcited by 1,653
  11. 2022Competition-Level Code Generation with AlphaCodecited by 1,462
  12. 2023GAIA: a benchmark for General AI Assistantscited by 1,128
19 more
  1. 2024Large Language Monkeys: Scaling Inference Compute with Repeated Samplingcited by 991
  2. 2023ChatDev: Communicative Agents for Software Developmentcited by 919
  3. 2023DIN-SQL: Decomposed In-Context Learning of Text-to-SQL with Self-Correctioncited by 477
  4. 2022CodeT: Code Generation with Generated Testscited by 384
  5. 2018Blockwise Parallel Decoding for Deep Autoregressive Modelscited by 323
  6. 2023MedAgents: Large Language Models as Collaborators for Zero-shot Medical Reasoningcited by 316
  7. 2023LLMLingua: Compressing Prompts for Accelerated Inference of Large Language Modelscited by 287
  8. 2025FlashInfer: Efficient and Customizable Attention Engine for LLM Inference Servingcited by 258
  9. 2023What can Large Language Models do in chemistry? A comprehensive benchmark on eight taskscited by 185
  10. 2023An LLM Compiler for Parallel Function Callingcited by 179
  11. 2023Skeleton-of-Thought: Prompting LLMs for Efficient Parallel Generationcited by 105
  12. 2023Batch Prompting: Efficient Inference with Large Language Model APIscited by 85
  13. 2024Planning In Natural Language Improves LLM Search For Code Generationcited by 83
  14. 2022Interactive Code Generation via Test-Driven User-Intent Formalizationcited by 50
  15. 2023Diversity of Thought Improves Reasoning Abilities of LLMscited by 25
  16. 2023CHORUS: Foundation Models for Unified Data Discovery and Explorationcited by 24
  17. 2024Scene Graph Generation with Role-Playing Large Language Modelscited by 16
  18. 2024DracoGPT: Extracting Visualization Design Preferences from Large Language Modelscited by 12
  19. 2024Self-Enhancing Video Data Management System for Compositional Events with Large Language Models [Technical Report]cited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 4; 2026: 10.

14 papers.

Sort:

By area:

  1. 2026The Evolution of Tool Use in LLM Agents: From Single-Tool Call to Multi-Tool Orchestrationcited by 14
  2. 2026DualSpec: Accelerating Deep Research Agents via Dual-Process Action Speculationcited by 8
  3. 2025When Should Users Check? Modeling Confirmation Frequency in Multi-Step Agentic AI Taskscited by 6
  4. 2026Beyond Individual Intelligence: Surveying Collaboration, Failure Attribution, and Self-Evolution in LLM-based Multi-Agent Systemscited by 6
  5. 2026DesignAsCode: Bridging Structural Editability and Visual Fidelity in Graphic Design Generationcited by 5
  6. 2025Agentic Inequalitycited by 3
  7. 2026Multi-Agent Reasoning Improves Compute Efficiency: Pareto-Optimal Test-Time Scalingcited by 3
  8. 2025Confident RAG: Enhancing the Performance of LLMs for Mathematics Question Answering through Multi-Embedding and Confidence Scoringcited by 2
  9. 2025Orchestrator: Active Inference for Multi-Agent Systems in Long-Horizon Taskscited by 2
  10. 2026Learning to Share: Selective Memory for Efficient Parallel Agentic Systemscited by 2
  11. 2026CONCAT: Consensus- and Confidence-Driven Ad Hoc Teaming for Efficient LLM-Based Multi-Agent Systemscited by 1
  12. 2026A Two-Tier Perspective on Inference-Time Parallelism in Multi-Agent LLM Systemscited by 1
2 more
  1. 2026Towards Direct Latent-Space Synthesis for Parallel Branches in LLM-Agent Workflowscited by 0
  2. 2026CIPHER: A Decoupled Exploration-Selection Framework for Test-Time Scaling of Data Science Agentscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025DynTaskMAS: A Dynamic Task Graph-driven Framework for Asynchronous and Parallel LLM-based Multi-Agent Systems
  2. 2025Plan-over-Graph: Towards Parallelable LLM Agent Schedule
  3. 2025Parallelism Meets Adaptiveness: Scalable Documents Understanding in Multi-Agent LLM Systems
  4. 2026Decentralized Multi-Agent Systems with Shared Context
  5. 2025Robotouille: An Asynchronous Planning Benchmark for LLM Agents
  6. 2025Hogwild! Inference: Parallel LLM Generation via Concurrent Attention
  7. 2026AgentCollab: A Self-Evaluation-Driven Collaboration Paradigm for Efficient LLM Agents
  8. 2026Learning What to Retain: Gated-Memory Routing for Efficient Collaboration in Multi-Agent LLM Systems