mlsanityData to 7 Oct 2026

When Cloud Agents Meet Device Agents: Lessons from Hybrid Multi-Agent Systems

arXiv 2605.30102 ↗First posted 28 May 2026cs.MA cs.AIcited by 0

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

The design space of agentic AI inference spans two extremes: frontier large language models (LLMs), typically hosted in the cloud and offering strong performance across a wide range of tasks at substantially high cost, and more cost-efficient small language models (SLMs), which are amenable to on-device inference. Hybrid multi-agent systems (MASs) combining on-device and cloud models offer a promising middle ground, but they also introduce a complex and poorly understood design space in which task accuracy, monetary cost, and edge energy consumption are tightly coupled; in the absence of general design principles, hybrid components, although not the most prevalent choice, are typically introduced through ad hoc decisions tailored to specific domains. In this work, we examine this design space more systematically. We adapt two representative MAS architectures to support hybrid inference and study how individual design choices shift the operating point along the Pareto frontier of power, cost, and performance. Our findings paint a nuanced picture of hybrid MAS design: while SLMs can effectively benefit from LLM assistance, the optimal architecture is highly task-dependent, and greater frontier-level compute does not consistently translate to better performance.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 51 of its 64 references that are in the map. First posted on arXiv 28 May 2026.

Builds on

51 of 64 references.

Closest ancestors · 14

Same direction, newest first

  1. 2025Small Language Models for Efficient Agentic Tool Calling: Outperforming Large Models with Targeted Fine-tuning17 Dec
  2. 2025Towards a Science of Scaling Agent Systems9 Dec
  3. 2025AgentArch: A Comprehensive Benchmark to Evaluate Agent Architectures in Enterprise13 Sep
  4. 2025Small Language Models are the Future of Agentic AI2 Jun
  5. 2025Distilling LLM Agent into Small Models with Retrieval and Code Tools23 May
  6. 2025AgentNet: Decentralized Evolutionary Coordination for LLM-based Multi-Agent Systems1 Apr
  7. 2025LLMs Working in Harmony: A Survey on the Technological Aspects of Building Effective LLM-Based Multi Agent Systems13 Mar
  8. 2025MultiAgentBench: Evaluating the Collaboration and Competition of LLM agents3 Mar
  9. 2025Multi-Agent Collaboration Mechanisms: A Survey of LLMs10 Jan
  10. 2024A Survey on LLM-based Multi-Agent System: Recent Advances and New Frontiers in Application23 Dec
  11. 2024Cut the Crap: An Economical Communication Pipeline for LLM-based Multi-Agent Systems3 Oct
  12. 2024Chain of Agents: Large Language Models Collaborating on Long-Context Tasks4 Jun
2 more
  1. 2024LLM Multi-Agent Systems: Challenges and Open Problems5 Feb
  2. 2024Large Language Model based Multi-Agents: A Survey of Progress and Challenges21 Jan

Other references · 37

  1. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  2. 2025Qwen3 Technical Reportcited by 8,195
  3. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  4. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  5. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
  6. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,838
  7. 2023Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizescited by 710
  8. 2024RouteLLM: Learning to Route LLMs with Preference Datacited by 633
  9. 2024KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantizationcited by 571
  10. 2025Why Do Multi-Agent LLM Systems Fail?cited by 559
  11. 2024AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agentscited by 359
  12. 2024Hybrid LLM: Cost-Efficient and Quality-Aware Query Routingcited by 354
25 more
  1. 2024Mobile-Agent-v2: Mobile Device Operation Assistant with Effective Navigation via Multi-Agent Collaborationcited by 187
  2. 2023LLM in a flash: Efficient Large Language Model Inference with Limited Memorycited by 170
  3. 2024Small Language Models: Survey, Measurements, and Insightscited by 109
  4. 2025MAI-UI Technical Report: Real-World Centric Foundation GUI Agentscited by 88
  5. 2025Universal Model Routing for Efficient LLM Inferencecited by 80
  6. 2025In-the-Flow Agentic System Optimization for Effective Planning and Tool Usecited by 67
  7. 2025Talk Isn't Always Cheap: Understanding Failure Modes in Multi-Agent Debatecited by 63
  8. 2025The Illusion of Diminishing Returns: Measuring Long Horizon Execution in LLMscited by 62
  9. 2026Memory for Autonomous LLM Agents:Mechanisms, Evaluation, and Emerging Frontierscited by 60
  10. 2024TensorOpera Router: A Multi-Model Router for Efficient LLM Inferencecited by 59
  11. 2025Scaling LLM Multi-turn RL with End-to-end Summarization-based Context Managementcited by 51
  12. 2024LoRA Land: 310 Fine-tuned LLMs that Rival GPT-4, A Technical Reportcited by 47
  13. 2025ToolOrchestra: Elevating Intelligence via Efficient Model and Tool Orchestrationcited by 44
  14. 2026UI-Venus-1.5 Technical Reportcited by 39
  15. 2024LLM Self-Correction with DeCRIM: Decompose, Critique, and Refine for Enhanced Following of Instructions with Multiple Constraintscited by 34
  16. 2024LLaMAR: Long-Horizon Planning for Multi-Agent Robots in Partially Observable Environmentscited by 32
  17. 2024FanOutQA: A Multi-Hop, Multi-Document Question Answering Benchmark for Large Language Modelscited by 29
  18. 2024DDK: Distilling Domain Knowledge for Efficient Large Language Modelscited by 25
  19. 2025How to Train Your Advisor: Steering Black-Box LLMs with Advisor Modelscited by 20
  20. 2024Efficient LLM Inference using Dynamic Input Pruning and Cache-Aware Maskingcited by 10
  21. 2024Privacy-preserved LLM Cascade via CoT-enhanced Policy Learningcited by 8
  22. 2025Benchmarking and Studying the LLM-based Agent System in End-to-End Software Developmentcited by 5
  23. 2026DR-Venus: Towards Frontier Edge-Scale Deep Research Agents with Only 10K Open Datacited by 5
  24. 2025Context Discipline and Performance Correlation: Analyzing LLM Performance and Quality Degradation Under Varying Context Lengthscited by 2
  25. 2026DUAL-BLADE: Dual-Path NVMe-Direct KV-Cache Offloading for Edge LLM Inferencecited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025HERA: Hybrid Edge-cloud Resource Allocation for Cost-Efficient AI Agents
  2. 2025Toward Super Agent System with Hybrid AI Routers
  3. 2025The Cost of Dynamic Reasoning: Demystifying AI Agents and Test-Time Scaling from an AI Infrastructure Perspective
  4. 2026Where Should Agents Live? Energy-Memory Characterization of Agentic AI for the Edge-Cloud Continuum
  5. 2026Heterogeneous Computing: The Key to Powering the Future of AI Agent Inference
  6. 2025Division-of-Thoughts: Harnessing Hybrid Language Model Synergy for Efficient On-Device Agents
  7. 2026AgBench: Agentic AI Benchmarks for Personal AI Devices
  8. 2026Hera: Learning Long-Horizon Coordination for Device-Cloud Collaborative LLM Agents