mlsanityData to 14 Oct 2026

Chain of Agents: Large Language Models Collaborating on Long-Context Tasks

arXiv 2406.02818 ↗First posted 4 Jun 2024cs.CLcited by 206

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

Addressing the challenge of effectively processing long contexts has become a critical issue for Large Language Models (LLMs). Two common strategies have emerged: 1) reducing the input length, such as retrieving relevant chunks by Retrieval-Augmented Generation (RAG), and 2) expanding the context window limit of LLMs. However, both strategies have drawbacks: input reduction has no guarantee of covering the part with needed information, while window extension struggles with focusing on the pertinent information for solving the task. To mitigate these limitations, we propose Chain-of-Agents (CoA), a novel framework that harnesses multi-agent collaboration through natural language to enable information aggregation and context reasoning across various LLMs over long-context tasks. CoA consists of multiple worker agents who sequentially communicate to handle different segmented portions of the text, followed by a manager agent who synthesizes these contributions into a coherent final output. CoA processes the entire input by interleaving reading and reasoning, and it mitigates long context focus issues by assigning each agent a short context. We perform comprehensive evaluation of CoA on a wide range of long-context tasks in question answering, summarization, and code completion, demonstrating significant improvements by up to 10% over strong baselines of RAG, Full-Context, and multi-agent LLMs.

Key facts. Cited by 206 papers in the map (data through 14 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 78 of its 92 references that are in the map. First posted on arXiv 4 Jun 2024.

Builds on

78 of 92 references.

Closest ancestors · 6

Same direction, newest first

  1. 2024LongAgent: Scaling Language Models to 128k Context through Multi-Agent Collaboration18 Feb
  2. 2024Large Language Model based Multi-Agents: A Survey of Progress and Challenges21 Jan
  3. 2023Towards Reasoning in Large Language Models via Multi-Agent Peer Review Collaboration14 Nov
  4. 2023Multi-Agent Consensus Seeking via Large Language Models31 Oct
  5. 2023Building Cooperative Embodied Agents Modularly with Large Language Models5 Jul
  6. 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society31 Mar

Other references · 72

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  5. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  6. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  7. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
  8. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,568
  9. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
  10. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
  11. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,998
  12. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,784
38 more
  1. 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,417
  2. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,407
  3. 2020Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Stepscited by 1,282
  4. 2022Text Embeddings by Weakly-Supervised Contrastive Pre-trainingcited by 1,253
  5. 2023PaLM 2 Technical Reportcited by 1,162
  6. 2021Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolationcited by 1,103
  7. 2022Atlas: Few-shot Learning with Retrieval Augmented Language Modelscited by 952
  8. 2021MuSiQue: Multihop Questions via Single-hop Question Compositioncited by 916
  9. 2023REPLUG: Retrieval-Augmented Black-Box Language Modelscited by 792
  10. 2023Extending Context Window of Large Language Models via Positional Interpolationcited by 740
  11. 2017The NarrativeQA Reading Comprehension Challengecited by 597
  12. 2022Decomposed Prompting: A Modular Approach for Solving Complex Taskscited by 558
  13. 2021A Dataset of Information-Seeking Questions and Answers Anchored in Research Paperscited by 486
  14. 2023LongLLMLingua: Accelerating and Enhancing LLMs in Long Context Scenarios via Prompt Compressioncited by 453
  15. 2021QMSum: A New Benchmark for Query-based Multi-domain Meeting Summarizationcited by 365
  16. 2023RepoBench: Benchmarking Repository-Level Code Auto-Completion Systemscited by 353
  17. 2021Efficient Attentions for Long Document Summarizationcited by 345
  18. 2023MedAgents: Large Language Models as Collaborators for Zero-shot Medical Reasoningcited by 316
  19. 2023ReConcile: Round-Table Conference Improves Reasoning via Consensus among Diverse LLMscited by 309
  20. 2022Social Simulacra: Creating Populated Prototypes for Social Computing Systemscited by 301
  21. 2024Long-context LLMs Struggle with Long In-context Learningcited by 286
  22. 2021AI Chains: Transparent and Controllable Human-AI Interaction by Chaining Large Language Model Promptscited by 268
  23. 2023Exploring Large Language Models for Communication Games: An Empirical Study on Werewolfcited by 241
  24. 2023Landmark Attention: Random-Access Infinite Context Length for Transformerscited by 213
  25. 2021QuALITY: Question Answering with Long Input Texts, Yes!cited by 208
  26. 2024Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attentioncited by 200
  27. 2021BookSum: A Collection of Datasets for Long-form Narrative Summarizationcited by 184
  28. 2023ZeroSCROLLS: A Zero-Shot Benchmark for Long Text Understandingcited by 176
  29. 2023RoCo: Dialectic Multi-Robot Collaboration with Large Language Modelscited by 172
  30. 2023Large Language Model Routing with Benchmark Datasetscited by 172
  31. 2022SCROLLS: Standardized CompaRison Over Long Language Sequencescited by 170
  32. 2023Language Agents with Reinforcement Learning for Strategic Play in the Werewolf Gamecited by 152
  33. 2023War and Peace (WarAgent): Large Language Model-based Multi-Agent Simulation of World Warscited by 145
  34. 2023Walking Down the Memory Maze: Beyond Context Limit through Interactive Readingcited by 133
  35. 2024Make Your LLM Fully Utilize the Contextcited by 119
  36. 2020Unsupervised Question Decomposition for Question Answeringcited by 118
  37. 2023How to Train Your DRAGON: Diverse Augmentation Towards Generalizable Dense Retrievalcited by 118
  38. 2023Retrieval meets Long Context Large Language Modelscited by 115

50 most cited of 72 All 72 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 9; 2025: 113; 2026: 84.

206 papers.

Sort:

By area:

  1. 2025Large Language Model Agent: A Survey on Methodology, Applications and Challengescited by 205
  2. 2025Scaling Long-Horizon LLM Agent via Context-Foldingcited by 148
  3. 2025AgenTracer: Who Is Inducing Failure in the LLM Agentic Systems?cited by 116
  4. 2025Multi-Agent Design: Optimizing Agents with Better Prompts and Topologiescited by 113
  5. 2024A Review of Large Language Models and Autonomous Agents in Chemistrycited by 99
  6. 2025Latent Collaboration in Multi-Agent Systemscited by 73
  7. 2024CompAct: Compressing Retrieved Documents Actively for Question Answeringcited by 64
  8. 2025Cache-to-Cache: Direct Semantic Communication Between Large Language Modelscited by 61
  9. 2025Learning Adaptive Parallel Reasoning with Language Modelscited by 58
  10. 2025LongReason: A Synthetic Long-Context Reasoning Benchmark via Context Expansioncited by 50
  11. 2025ContextAgent: Context-Aware Proactive LLM Agents with Open-World Sensory Perceptionscited by 45
  12. 2025Understanding the Information Propagation Effects of Communication Topologies in LLM-based Multi-Agent Systemscited by 43
38 more
  1. 2025Vibe Coding vs. Agentic Coding: Fundamentals and Practical Implications of Agentic AIcited by 42
  2. 2025Towards Agentic RAG with Deep Reasoning: A Survey of RAG-Reasoning Systems in LLMscited by 42
  3. 2024SuffixDecoding: Extreme Speculative Decoding for Emerging AI Applicationscited by 36
  4. 2025Optimizing Model Selection for Compound AI Systemscited by 36
  5. 2025From Language to Action: A Review of Large Language Models as Autonomous Agents and Tool Userscited by 32
  6. 2025Agentic Plan Caching: Test-Time Memory for Fast and Cost-Efficient LLM Agentscited by 27
  7. 2024Compute Or Load KV Cache? Why Not Both?cited by 25
  8. 2025LongRoPE2: Near-Lossless LLM Context Window Scalingcited by 24
  9. 2025Heterogeneous Swarms: Jointly Optimizing Model Roles and Weights for Multi-LLM Systemscited by 23
  10. 2025AI Scientists Fail Without Strong Implementation Capabilitycited by 21
  11. 2025Minions: Cost-efficient Collaboration Between On-device and Cloud Language Modelscited by 20
  12. 2025Resolving Evidence Sparsity: Agentic Context Engineering for Long-Document Understandingcited by 18
  13. 2024Plurals: A System for Guiding LLMs Via Simulated Social Ensemblescited by 17
  14. 2024LLM×MapReduce: Simplified Long-Sequence Processing using Large Language Modelscited by 17
  15. 2024A Systematic Assessment of OpenAI o1-Preview for Higher Order Thinking in Educationcited by 16
  16. 2025GINGER: Grounded Information Nugget-Based Generation of Responsescited by 16
  17. 2026Effective Strategies for Asynchronous Software Engineering Agentscited by 16
  18. 2025Self-Resource Allocation in Multi-Agent LLM Systemscited by 15
  19. 2025CoRT: Code-integrated Reasoning within Thinkingcited by 15
  20. 2025How Far Are AI Scientists from Changing the World?cited by 15
  21. 2025Uncertainty-Aware GUI Agent: Adaptive Perception through Component Recommendation and Human-in-the-Loop Refinementcited by 15
  22. 2025MARS-SQL: A multi-agent reinforcement learning framework for Text-to-SQLcited by 15
  23. 2026LatentMem: Customizing Latent Memory for Multi-Agent Systemscited by 15
  24. 2026LongRLVR: Long-Context Reinforcement Learning Requires Verifiable Context Rewardscited by 15
  25. 2025A Survey on Collaborative Mechanisms Between Large and Small Language Modelscited by 14
  26. 2026Agentic Artificial Intelligence (AI): Architectures, Taxonomies, and Evaluation of Large Language Model Agentscited by 14
  27. 2025ReAgent: Reversible Multi-Agent Reasoning for Knowledge-Enhanced Multi-Hop QAcited by 13
  28. 2025Environment Scaling for Interactive Agentic Experience Collection: A Surveycited by 13
  29. 2025Position: The Real Barrier to LLM Agent Usability is Agentic ROIcited by 12
  30. 2025Seven Security Challenges in Cross-domain Multi-agent LLM Systemscited by 12
  31. 2025From Reproduction to Replication: Evaluating Research Agents with Progressive Code Maskingcited by 12
  32. 2025How to Train a Leader: Hierarchical Reasoning in Multi-Agent LLMscited by 12
  33. 2025SafeSieve: From Heuristics to Experience in Progressive Pruning for LLM-based Multi-Agent Communicationcited by 12
  34. 2026VitaBench 2.0: Evaluating Personalized and Proactive Agents in Long-Term User Interactionscited by 12
  35. 2025Eliciting In-context Retrieval and Reasoning for Long-context Large Language Modelscited by 11
  36. 2025Game Theory Meets LLM and Agentic AI: Reimagining Cybersecurity for the Age of Intelligent Threatscited by 11
  37. 2025Self-Taught Agentic Long Context Understandingcited by 10
  38. 2025LLMs Working in Harmony: A Survey on the Technological Aspects of Building Effective LLM-Based Multi Agent Systemscited by 10

50 most cited of 206 All 206 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Tree of Agents: Improving Long-Context Capabilities of Large Language Models through Multi-Perspective Reasoning
  2. 2026Coding Agents are Effective Long-Context Processors
  3. 2026Can Small Agents Collaborate to Beat a Single Large Language Model?
  4. 2024GraphReader: Building Graph-based Agent to Enhance Long-Context Abilities of Large Language Models
  5. 2025RAGCap-Bench: Benchmarking Capabilities of LLMs in Agentic Retrieval Augmented Generation Systems
  6. 2026AgentLongBench: A Controllable Long Benchmark For Long-Contexts Agents via Environment Rollouts
  7. 2025Talk Structurally, Act Hierarchically: A Collaborative Framework for LLM Multi-Agent Systems
  8. 2024Dynamic Multi-Agent Orchestration and Retrieval for Multi-Source Question-Answer Systems using Large Language Models