mlsanityData to 14 Oct 2026

Efficient Agents: Building Effective Agents While Reducing Cost

arXiv 2508.02694 ↗First posted 24 Jul 2025cs.AI cs.CL cs.MAcited by 32

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

The remarkable capabilities of Large Language Model (LLM)-driven agents have enabled sophisticated systems to tackle complex, multi-step tasks, but their escalating costs threaten scalability and accessibility. This work presents the first systematic study of the efficiency-effectiveness trade-off in modern agent systems, addressing the critical need for cost-effective designs without sacrificing performance. We investigate three key questions: (1) How much complexity do agentic tasks inherently require? (2) When do additional modules yield diminishing returns? (3) How much efficiency can be gained through the design of efficient agent frameworks? Through an empirical analysis on the GAIA benchmark, we evaluate the impact of LLM backbone selection, agent framework designs, and test-time scaling strategies. Using the cost-of-pass metric, we quantify the efficiency-performance trade-off across these dimensions. Our findings inform the development of Efficient Agents , a novel agent framework that has an optimal complexity to task requirements. Efficient Agents retains 96.7% of the performance of OWL, one leading open-source agent framework, while reducing operational costs from 0.398 to 0.228, resulting in a 28.4% improvement in cost-of-pass. Our work provides actionable insights for designing efficient, high-performing agent systems, advancing the accessibility and sustainability of AI-driven solutions.

Key facts. Cited by 32 papers in the map (data through 14 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 32 of its 41 references that are in the map. First posted on arXiv 24 Jul 2025.

Builds on

32 of 41 references.

Closest ancestors · 7

Same direction, newest first

  1. 2025Scaling Test-time Compute for LLM Agents15 Jun
  2. 2025OWL: Optimized Workforce Learning for General Multi-Agent Assistance in Real-World Task Automation29 May
  3. 2024BudgetMLAgent: A Cost-Effective LLM Multi-Agent system for Automating Machine Learning Tasks12 Nov
  4. 2024Cut the Crap: An Economical Communication Pipeline for LLM-based Multi-Agent Systems3 Oct
  5. 2024Symbolic Learning Enables Self-Evolving Agents26 Jun
  6. 2024Understanding the planning of LLM agents: A survey5 Feb
  7. 2023Agents: An Open-source Framework for Autonomous Language Agents14 Sep

Other references · 25

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  4. 2025Qwen3 Technical Reportcited by 8,820
  5. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  6. 2020Scaling Laws for Neural Language Modelscited by 6,586
  7. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  8. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,307
  9. 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,988
  10. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,986
  11. 2024Large Language Monkeys: Scaling Inference Compute with Repeated Samplingcited by 991
  12. 2025BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agentscited by 647
13 more
  1. 2024A Survey on the Memory Mechanism of Large Language Model based Agentscited by 634
  2. 2024Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMscited by 570
  3. 2020BERT Loses Patience: Fast and Robust Inference with Early Exitcited by 273
  4. 2024Token-Budget-Aware LLM Reasoningcited by 247
  5. 2020BERT-of-Theseus: Compressing BERT by Progressive Module Replacingcited by 113
  6. 2021BERT Learns to Teach: Knowledge Distillation with Meta Learningcited by 61
  7. 2022EfficientVLM: Fast and Accurate Vision-Language Models via Knowledge Distillation and Modal-adaptive Pruningcited by 56
  8. 2024Reasoning in Token Economies: Budget-Aware Evaluation of LLM Reasoning Strategiescited by 45
  9. 2021Beyond Preserved Accuracy: Evaluating Loyalty and Robustness of BERT Compressioncited by 38
  10. 2025Cost-of-Pass: An Economic Framework for Evaluating Language Modelscited by 35
  11. 2023Efficient Prompting via Dynamic In-Context Learningcited by 18
  12. 2025UORA: Uniform Orthogonal Reinitialization Adaptation in Parameter-Efficient Fine-Tuning of Large Modelscited by 11
  13. 2023Modular Transformers: Compressing Transformers into Modularized Layers for Flexible Efficient Inferencecited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 8; 2026: 24.

32 papers.

Sort:

By area:

  1. 2025ToolOrchestra: Elevating Intelligence via Efficient Model and Tool Orchestrationcited by 47
  2. 2025Budget-Aware Tool Use Enables Effective Agent Scalingcited by 44
  3. 2026How Do AI Agents Spend Your Money? Analyzing and Predicting Token Consumption in Agentic Coding Taskscited by 33
  4. 2025Flash-Searcher: Fast and Effective Web Agents via DAG-Based Parallel Executioncited by 27
  5. 2025Towards Personalized Deep Research: Benchmarks and Evaluationscited by 16
  6. 2026Toward Efficient Agents: Memory, Tool learning, and Planningcited by 16
  7. 2026Inference-Time Budget Control for LLM Search Agentscited by 13
  8. 2025Stop Wasting Your Tokens: Towards Efficient Runtime Multi-Agent Systemscited by 12
  9. 2025How Far Are We from Genuinely Useful Deep Research Agents?cited by 11
  10. 2025TPS-Bench: Evaluating AI Agents' Tool Planning & Scheduling Abilities in Compounding Taskscited by 9
  11. 2026AtlasVA: Self-Evolving Visual Skill Memory for Teacher-Free VLM Agentscited by 7
  12. 2026Why Search When You Can Transfer? Amortized Agentic Workflow Design from Structural Priorscited by 6
20 more
  1. 2026Agent-Omit: Adaptive Context Omission for Efficient LLM Agentscited by 5
  2. 2026PreFlect: From Retrospective to Prospective Reflection in Large Language Model Agentscited by 4
  3. 2026Beyond Accuracy: Unveiling Inefficiency Patterns in Tool-Integrated Reasoningcited by 3
  4. 2026Efficient Agent Evaluation via Diversity-Guided User Simulationcited by 3
  5. 2026Only Ask What You Don't Know: Grounded Delta Planning for Efficient Multi-step RAGcited by 3
  6. 2026When Agents Coordinate: Measuring Coordination in Multi-Agent AI Codingcited by 3
  7. 2025Reliable agent engineering should integrate machine-compatible organizational principlescited by 2
  8. 2026Cost and Accuracy of Long-Term Memory in Distributed Multi-Agent Systems Based on Large Language Modelscited by 2
  9. 2026Last Step Matters: Early Uncertainty Cannot Predict Failure in Long-Horizon Agentscited by 2
  10. 2026CAM: A Causality-based Analysis Framework for Multi-Agent Code Generation Systemscited by 1
  11. 2026AgentSpec: Speculative Decoding for Batch Inference of LLM Agentscited by 1
  12. 2026The Missing Piece in Pre-trained Model Evaluation: Reward-Guided Decoding Unlocks Task-Oriented Behavior Without Parameter Updatescited by 0
  13. 2026Can Agentic Trading Systems Pay for Their Own Intelligence?cited by 0
  14. 2026The Hidden Footprint: Making Storage a First-Class Metric for LLM Agent Evaluationcited by 0
  15. 2026OpenCodeReview: Determinism over Non-Determinism for Cost-Effective Agent-Based Code Reviewcited by 0
  16. 2026Don't Overthink, Don't Underthink: Toward Adaptive Reasoning in Agentic AIcited by 0
  17. 2026Defining AI Agents: A Compendium of Criteria, Metrics, and Benchmarkscited by 0
  18. 2026LIMBO: Lifelong Inference-Time Memory and Budget Optimization for LLM Agentscited by 0
  19. 2026LEGIT: Credentialing Protocol for Trustworthy AI Agent Marketplacescited by 0
  20. 2026AgentBetta: Verification-Driven Adaptive Configuration of an AI Nano-Agent through Selective Expansion and Verified Contractioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Rethinking the Evaluation of Efficiency Methods for Multi-Agent Systems
  2. 2025AgentSwift: Efficient LLM Agent Design via Value-guided Hierarchical Search
  3. 2025What Limits Agentic Systems Efficiency?
  4. 2026AgentFactory: Towards Automated Agentic System Design and Optimization
  5. 2026AgentGym2: Benchmarking Large Language Model Agents in De-Idealized Real-World Environments
  6. 2026Utility-Guided Agent Orchestration for Efficient LLM Tool Use
  7. 2024AgentSquare: Automatic LLM Agent Search in Modular Design Space
  8. 2024AgentQuest: A Modular Benchmark Framework to Measure Progress and Improve LLM Agents