mlsanityData to 7 Oct 2026

HERA: Hybrid Edge-cloud Resource Allocation for Cost-Efficient AI Agents

arXiv 2504.00434 ↗First posted 1 Apr 2025cs.LGcited by 1

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

In the realm of AI, large language models (LLMs) like GPT-4, central to the operation of AI agents, predominantly operate in the cloud, incurring high operational costs. With local-based small language models (SLMs) becoming more accurate, the necessity of cloud-exclusive processing is being reconsidered. An AI agent's response to a user's request comprises a series of subtasks or iterations. Existing approaches only allocate a single request between SLM and LLM to ensure their outputs are similar, but adopting this approach in the AI agent scenario for assigning each subtask is not effective since SLM will output a different subsequent subtask, which affects the accuracy of the final output. In this paper, we first conduct experimental analysis to understand the features of AI agent operations. Leveraging our findings, we propose the Adaptive Iteration-level Model Selector (AIMS), a lightweight scheduler to automatically partition AI agent's subtasks between local-based SLM and cloud-based LLM. AIMS considers the varying subtask features and strategically decides the location for each subtask in order to use SLM as much as possible while attaining the accuracy level. Our experimental results demonstrate that AIMS increases accuracy by up to 9.1% and SLM usage by up to 10.8% compared to HybridLLM. It offloads 45.67% of subtasks to a local SLM while attaining similar accuracy on average compared with the cloud-only LLM approach.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 50 of its 62 references that are in the map. First posted on arXiv 1 Apr 2025.

Builds on

50 of 62 references.

Closest ancestors · 6

Same direction, newest first

  1. 2024Automated Design of Agentic Systems15 Aug
  2. 2023Balancing Autonomy and Alignment: A Multi-Dimensional Taxonomy for Autonomous LLM-powered Multi-Agent Architectures5 Oct
  3. 2023Adapting LLM Agents with Universal Feedback in Communication1 Oct
  4. 2023The Rise and Potential of Large Language Model Based Agents: A Survey14 Sep
  5. 2023Cognitive Architectures for Language Agents5 Sep
  6. 2023A Survey on Large Language Model based Autonomous Agents22 Aug

Other references · 44

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  5. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  6. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  7. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  8. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  9. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  10. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  11. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  12. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
32 more
  1. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  2. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  3. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  4. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  5. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  6. 2023Mistral 7Bcited by 3,065
  7. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
  8. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,722
  9. 2021DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharingcited by 1,219
  10. 2022WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agentscited by 1,197
  11. 2023HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Facecited by 1,173
  12. 2019DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphscited by 883
  13. 2023FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performancecited by 791
  14. 2023Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizescited by 710
  15. 2024RouteLLM: Learning to Route LLMs with Preference Datacited by 633
  16. 2023LLM-Blender: Ensembling Large Language Models with Pairwise Ranking and Generative Fusioncited by 550
  17. 2023The Falcon Series of Open Language Modelscited by 526
  18. 2023Augmented Language Models: a Surveycited by 398
  19. 2023SpecInfer: Accelerating Generative Large Language Model Serving with Tree-based Speculative Inference and Verificationcited by 359
  20. 2024Hybrid LLM: Cost-Efficient and Quality-Aware Query Routingcited by 354
  21. 2023Chameleon: Plug-and-Play Compositional Reasoning with Large Language Modelscited by 352
  22. 2023Ghost in the Minecraft: Generally Capable Agents for Open-World Environments via Large Language Models with Text-based Knowledge and Memorycited by 288
  23. 2023Specializing Smaller Language Models towards Multi-Step Reasoningcited by 249
  24. 2022Making Large Language Models Better Reasoners with Step-Aware Verifiercited by 231
  25. 2023TaskMatrix.AI: Completing Tasks by Connecting Foundation Models with Millions of APIscited by 184
  26. 2023ReWOO: Decoupling Reasoning from Observations for Efficient Augmented Language Modelscited by 174
  27. 2023SpotServe: Serving Generative Large Language Models on Preemptible Instancescited by 83
  28. 2024NeuPIMs: NPU-PIM Heterogeneous Acceleration for Batched LLM Inferencingcited by 48
  29. 2024ExeGPT: Constraint-Aware Resource Scheduling for LLM Inferencecited by 43
  30. 2023Small Models are Valuable Plug-ins for Large Language Modelscited by 42
  31. 2021MetaQA: Combining Expert Agents for Multi-Skill Question Answeringcited by 15
  32. 2022On Efficient Approximate Queries over Machine Learning Modelscited by 7

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025Automating High Energy Physics Data Analysis with LLM-Powered Agentscited by 10

Similar, not cited

Close in wording, no citation either way

  1. 2026When Cloud Agents Meet Device Agents: Lessons from Hybrid Multi-Agent Systems
  2. 2025Collaborative Inference and Learning between Edge SLMs and Cloud LLMs: A Survey of Algorithms, Execution, and Open Challenges
  3. 2026Hera: Learning Long-Horizon Coordination for Device-Cloud Collaborative LLM Agents
  4. 2025Throughput-Optimal Scheduling Algorithms for LLM Inference and AI Agents
  5. 2025Division-of-Thoughts: Harnessing Hybrid Language Model Synergy for Efficient On-Device Agents
  6. 2025Adaptive AI Agent Placement and Migration in Edge Intelligence Systems
  7. 2024AdaSwitch: Adaptive Switching between Small and Large Agents for Effective Cloud-Local Collaborative Learning
  8. 2026End-to-End Latency-Minimizing and Load-Balanced Request Scheduling for Edge LLM Inference in Agentic AI Services