# LongAgent: Scaling Language Models to 128k Context through Multi-Agent Collaboration

arXiv: https://arxiv.org/abs/2402.11550 · HTML: https://mlsanity.com/paper/2402.11550/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use.
- Builds on 17 of its 34 references that are in the map.
- First posted on arXiv 18 Feb 2024.

## Placed in

- LLM agents and tool use / [General LLM agent frameworks and benchmarks](https://mlsanity.com/agents/general-llm-agent-frameworks-and-benchmarks/)

## Builds on (most cited first, 17 of 17)

- [Reflexion: Language Agents with Verbal Reinforcement Learning](https://mlsanity.com/paper/2303.11366/) (2023)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering](https://mlsanity.com/paper/1809.09600/) (2018)
- [Lost in the Middle: How Language Models Use Long Contexts](https://mlsanity.com/paper/2307.03172/) (2023)
- [Efficient Streaming Language Models with Attention Sinks](https://mlsanity.com/paper/2309.17453/) (2023)
- [Improving Factuality and Reasoning in Language Models through Multiagent Debate](https://mlsanity.com/paper/2305.14325/) (2023)
- [Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation](https://mlsanity.com/paper/2108.12409/) (2021)
- [Extending Context Window of Large Language Models via Positional Interpolation](https://mlsanity.com/paper/2306.15595/) (2023)
- [Large Language Models as Tool Makers](https://mlsanity.com/paper/2305.17126/) (2023)
- [Unleashing the Emergent Cognitive Synergy in Large Language Models: A Task-Solving Agent through Multi-Persona Self-Collaboration](https://mlsanity.com/paper/2307.05300/) (2023)
- [Landmark Attention: Random-Access Infinite Context Length for Transformers](https://mlsanity.com/paper/2305.16300/) (2023)
- [LLM Maybe LongLM: Self-Extend LLM Context Window Without Tuning](https://mlsanity.com/paper/2401.01325/) (2024)
- [Focused Transformer: Contrastive Training for Context Scaling](https://mlsanity.com/paper/2307.03170/) (2023)
- [Walking Down the Memory Maze: Beyond Context Limit through Interactive Reading](https://mlsanity.com/paper/2310.05029/) (2023)
- [Long Context Compression with Activation Beacon](https://mlsanity.com/paper/2401.03462/) (2024)
- [RecurrentGPT: Interactive Generation of (Arbitrarily) Long Text](https://mlsanity.com/paper/2305.13304/) (2023)
- [PEARL: Prompting Large Language Models to Plan and Execute Actions Over Long Documents](https://mlsanity.com/paper/2305.14564/) (2023)

## Built on by (most cited first, 30 of 38)

- [Chain of Agents: Large Language Models Collaborating on Long-Context Tasks](https://mlsanity.com/paper/2406.02818/) (2024)
- [Scaling Long-Horizon LLM Agent via Context-Folding](https://mlsanity.com/paper/2510.11967/) (2025)
- [MAPoRL: Multi-Agent Post-Co-Training for Collaborative Large Language Models with Reinforcement Learning](https://mlsanity.com/paper/2502.18439/) (2025)
- [Lifelong Learning of Large Language Model based Agents: A Roadmap](https://mlsanity.com/paper/2501.07278/) (2025)
- [LVAgent: Long Video Understanding by Multi-Round Dynamical Collaboration of MLLM Agents](https://mlsanity.com/paper/2503.10200/) (2025)
- [VCA: Video Curious Agent for Long Video Understanding](https://mlsanity.com/paper/2412.10471/) (2024)
- [Multimodal Needle in a Haystack: Benchmarking Long-Context Capability of Multimodal Large Language Models](https://mlsanity.com/paper/2406.11230/) (2024)
- [A Survey on LLM-based Multi-Agent System: Recent Advances and New Frontiers in Application](https://mlsanity.com/paper/2412.17481/) (2024)
- [LongReason: A Synthetic Long-Context Reasoning Benchmark via Context Expansion](https://mlsanity.com/paper/2501.15089/) (2025)
- [VideoChat-A1: Thinking with Long Videos by Chain-of-Shot Reasoning](https://mlsanity.com/paper/2506.06097/) (2025)
- [VideoARM: Agentic Reasoning over Hierarchical Memory for Long-Form Video Understanding](https://mlsanity.com/paper/2512.12360/) (2025)
- [Can We Rely on LLM Agents to Draft Long-Horizon Plans? Let's Take TravelPlanner as an Example](https://mlsanity.com/paper/2408.06318/) (2024)
- [LLM×MapReduce: Simplified Long-Sequence Processing using Large Language Models](https://mlsanity.com/paper/2410.09342/) (2024)
- [LongRLVR: Long-Context Reinforcement Learning Requires Verifiable Context Rewards](https://mlsanity.com/paper/2603.02146/) (2026)
- [Eliciting In-context Retrieval and Reasoning for Long-context Large Language Models](https://mlsanity.com/paper/2501.08248/) (2025)
- [Retrieval Meets Reasoning: Dynamic In-Context Editing for Long-Text Understanding](https://mlsanity.com/paper/2406.12331/) (2024)
- [When Does Divide and Conquer Work for Long Context LLM? A Noise Decomposition Framework](https://mlsanity.com/paper/2506.16411/) (2025)
- [MetaGen: Self-Evolving Roles and Topologies for Multi-Agent LLM Reasoning](https://mlsanity.com/paper/2601.19290/) (2026)
- [NeedleInATable: Exploring Long-Context Capability of Large Language Models towards Long-Structured Tables](https://mlsanity.com/paper/2504.06560/) (2025)
- [Tiered Agentic Oversight: A Hierarchical Multi-Agent System for Healthcare Safety](https://mlsanity.com/paper/2506.12482/) (2025)
- [MCRanker: Generating Diverse Criteria On-the-Fly to Improve Point-wise LLM Rankers](https://mlsanity.com/paper/2404.11960/) (2024)
- [Long Context Scaling: Divide and Conquer via Multi-Agent Question-driven Collaboration](https://mlsanity.com/paper/2505.20625/) (2025)
- [VideoChat-M1: Collaborative Policy Planning for Video Understanding via Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2511.19524/) (2025)
- [MASR: Self-Reflective Reasoning through Multimodal Hierarchical Attention Focusing for Agent-based Video Understanding](https://mlsanity.com/paper/2504.17213/) (2025)
- [The Cognitive Bandwidth Bottleneck: Shifting Long-Horizon Agent from Planning with Actions to Planning with Schemas](https://mlsanity.com/paper/2510.07091/) (2025)
- [SwarmSys: Decentralized Swarm-Inspired Agents for Scalable and Adaptive Reasoning](https://mlsanity.com/paper/2510.10047/) (2025)
- [SWIRL: A Staged Workflow for Interleaved Reinforcement Learning in Mobile GUI Control](https://mlsanity.com/paper/2508.20018/) (2025)
- [GCAgent: Long-Video Understanding via Schematic and Narrative Episodic Memory](https://mlsanity.com/paper/2511.12027/) (2025)
- [APB: Accelerating Distributed Long-Context Inference by Passing Compressed Context Blocks across GPUs](https://mlsanity.com/paper/2502.12085/) (2025)
- [BELLE: A Bi-Level Multi-Agent Reasoning Framework for Multi-Hop Question Answering](https://mlsanity.com/paper/2505.11811/) (2025)

## Abstract (arXiv)

Large language models (LLMs) have demonstrated impressive performance in understanding language and executing complex reasoning tasks. However, LLMs with long context windows have been notorious for their expensive training costs and high inference latency. Even the most advanced models such as GPT-4 and Claude2 often make mistakes when processing inputs of over 100k tokens, a phenomenon also known as textitlost in the middle. In this paper, we propose textscLongAgent, a method based on multi-agent collaboration, which scales LLMs (e.g., LLaMA) to a context of 128K and demonstrates potential superiority in long-text processing compared to GPT-4. In textscLongAgent, a leader is responsible for understanding user intent and directing team members to acquire information from documents. Due to members' hallucinations, it is non-trivial for a leader to obtain accurate information from the responses of dozens to hundreds of members. To address this, we develop an textitinter-member communication mechanism to resolve response conflicts caused by hallucinations through information sharing. Our experimental results indicate that textscLongAgent offers a promising alternative for long-text processing. The agent team instantiated with LLaMA-7B achieves significant improvements in tasks such as 128k-long text retrieval, multi-hop question answering, compared to GPT-4.