mlsanityData to 7 Oct 2026

Minions: Cost-efficient Collaboration Between On-device and Cloud Language Models

arXiv 2502.15964 ↗First posted 21 Feb 2025cs.LG cs.AI cs.CLcited by 17

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

We investigate an emerging setup in which a small, on-device language model (LM) with access to local data communicates with a frontier, cloud-hosted LM to solve real-world tasks involving financial, medical, and scientific reasoning over long documents. Can a local-remote collaboration reduce cloud inference costs while preserving quality? First, we consider a naive collaboration protocol where the local and remote models simply chat back and forth. Because only the local model reads the full context, this protocol achieves a 30.4x reduction in remote costs, but recovers only 87% of the performance of the frontier model. We identify two key limitations of this protocol: the local model struggles to (1) follow the remote model's multi-step instructions and (2) reason over long contexts. Motivated by these observations, we study an extension of this protocol, coined MinionS, in which the remote model decomposes the task into easier subtasks over shorter chunks of the document, that are executed locally in parallel. MinionS reduces costs by 5.7x on average while recovering 97.9% of the performance of the remote model alone. Our analysis reveals several key design choices that influence the trade-off between cost and performance in local-remote systems.

Key facts. Cited by 17 papers in the map (data through 7 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 44 of its 59 references that are in the map. First posted on arXiv 21 Feb 2025.

Builds on

44 of 59 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024PRISM: Efficient Long-Range Reasoning With Short-Context LLMs25 Dec
  2. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters6 Aug
  3. 2024Inference Scaling Laws: An Empirical Analysis of Compute-Optimal Inference for Problem-Solving with Language Models1 Aug

Other references · 41

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  3. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  4. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
  5. 2019Language Models as Knowledge Bases?cited by 1,922
  6. 2022Fast Inference from Transformers via Speculative Decodingcited by 1,615
  7. 2023MemGPT: Towards LLMs as Operating Systemscited by 1,295
  8. 2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,203
  9. 2024Large Language Monkeys: Scaling Inference Compute with Repeated Samplingcited by 946
  10. 2023DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelinescited by 803
  11. 2023FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performancecited by 791
  12. 2024Large Language Model based Multi-Agents: A Survey of Progress and Challengescited by 752
29 more
  1. 2019Latent Retrieval for Weakly Supervised Open Domain Question Answeringcited by 691
  2. 2021Retrieval Augmentation Reduces Hallucination in Conversationcited by 691
  3. 2021A Dataset of Information-Seeking Questions and Answers Anchored in Research Paperscited by 475
  4. 2024Mixture-of-Agents Enhances Large Language Model Capabilitiescited by 420
  5. 2022Text and Code Embeddings by Contrastive Pre-Trainingcited by 358
  6. 2021AI Chains: Transparent and Controllable Human-AI Interaction by Chaining Large Language Model Promptscited by 265
  7. 2023FinanceBench: A New Benchmark for Financial Question Answeringcited by 261
  8. 2024TextGrad: Automatic "Differentiation" via Textcited by 250
  9. 2024BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystackcited by 211
  10. 2024Chain of Agents: Large Language Models Collaborating on Long-Context Taskscited by 201
  11. 2022Ask Me Anything: A simple strategy for prompting language modelscited by 155
  12. 2023BooookScore: A systematic exploration of book-length summarization in the era of LLMscited by 137
  13. 2023Chain of Code: Reasoning with a Language Model-Augmented Code Emulatorcited by 120
  14. 2024The Good, The Bad, and The Greedy: Evaluation of LLMs Should Not Ignore Non-Determinismcited by 113
  15. 2024Sequoia: Scalable, Robust, and Hardware-aware Speculative Decodingcited by 103
  16. 2023Language Models Enable Simple Systems for Generating Structured Views of Heterogeneous Data Lakescited by 101
  17. 2024OpenELM: An Efficient Language Model Family with Open Training and Inference Frameworkcited by 74
  18. 2024Archon: An Architecture Search Framework for Inference-Time Techniquescited by 59
  19. 2024PAPILLON: Privacy Preservation from Internet-based and Local Language Model Ensemblescited by 54
  20. 2024DocETL: Agentic Query Rewriting and Evaluation for Complex Document Processingcited by 49
  21. 2024The Larger the Better? Improved LLM Code-Generation via Budget Reallocationcited by 41
  22. 2022Is a Question Decomposition Unit All We Need?cited by 37
  23. 2024LongHealth: A Question Answering Benchmark with Long Clinical Documentscited by 33
  24. 2024CoGenesis: A Framework Collaborating Large and Small Language Models for Secure Context-Aware Instruction Followingcited by 32
  25. 2024CE-CoLLM: Efficient and Adaptive Large Language Models Through Cloud-Edge Collaborationcited by 25
  26. 2024PerLLM: Personalized Inference Scheduling with Edge-Cloud Collaboration for Diverse LLM Servicescited by 23
  27. 2024Fast and Slow Generating: An Empirical Study on Large and Small Language Models Collaborative Decodingcited by 12
  28. 2024Writing in the Margins: Better Inference Pattern for Long Context Retrievalcited by 7
  29. 2024PhoneLM:an Efficient and Capable Small Language Model Family through Principled Pre-trainingcited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 9; 2026: 8.

17 papers.

Sort:

By area:

  1. 2025Cartridges: Lightweight and general-purpose long context representations via self-studycited by 74
  2. 2026Parcae: Scaling Laws For Stable Looped Language Modelscited by 45
  3. 2025Agentic Plan Caching: Test-Time Memory for Fast and Cost-Efficient LLM Agentscited by 23
  4. 2025Guiding Reasoning in Small Language Models with LLM Assistancecited by 13
  5. 2025Intelligence per Watt: Measuring Intelligence Efficiency of Local AIcited by 13
  6. 2025A Survey on Collaborative Mechanisms Between Large and Small Language Modelscited by 12
  7. 2025An Information Theoretic Perspective on Agentic System Designcited by 8
  8. 2025KAME: Tandem Architecture for Enhancing Knowledge in Real-Time Speech-to-Speech Conversational AIcited by 6
  9. 2025Bridging On-Device and Cloud LLMs for Collaborative Reasoning: A Unified Methodology for Local Routing and Post-Trainingcited by 4
  10. 2026Recursive Agent Harnessescited by 4
  11. 2026Hera: Learning Long-Horizon Coordination for Device-Cloud Collaborative LLM Agentscited by 3
  12. 2025Boomerang Distillation Enables Zero-Shot Model Size Interpolationcited by 2
5 more
  1. 2026OpenJarvis: Personal AI, On Personal Devicescited by 2
  2. 2026Divide-and-Conquer CoT: RL for Reducing Latency via Parallel Reasoningcited by 1
  3. 2026Evaluating Temporal Semantic Caching and Workflow Optimization in Agentic Plan-Execute Pipelinescited by 1
  4. 2026CONCORD: Asynchronous Sparse Aggregation for Device-Cloud RAG under Document Isolationcited by 0
  5. 2026Ask Without Telling: Local SLMs Consult Cloud LLMs Without Revealing Task Intentcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Collaborative Inference and Learning between Edge SLMs and Cloud LLMs: A Survey of Algorithms, Execution, and Open Challenges
  2. 2025A Novel Hat-Shaped Device-Cloud Collaborative Inference Framework for Large Language Models
  3. 2025A Survey on Collaborating Small and Large Language Models for Performance, Cost-effectiveness, Cloud-edge Privacy, and Trustworthiness
  4. 2025Large-Small Model Collaboration for Enhancing Edge-Deployed Small Models
  5. 2024EdgeShard: Efficient LLM Inference via Collaborative Edge Computing
  6. 2024AdaSwitch: Adaptive Switching between Small and Large Agents for Effective Cloud-Local Collaborative Learning
  7. 2025Synera: Synergistic LLM Serving across Device and Cloud at Scale
  8. 2026PyroDash: Cost-Efficient Token-Level Small-Large Language Model Collaborative Inference