# Efficient Agents: Building Effective Agents While Reducing Cost

arXiv: https://arxiv.org/abs/2508.02694 · HTML: https://mlsanity.com/paper/2508.02694/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use.
- Builds on 32 of its 41 references that are in the map.
- First posted on arXiv 24 Jul 2025.

## Placed in

- LLM agents and tool use / [General LLM agent frameworks and benchmarks](https://mlsanity.com/agents/general-llm-agent-frameworks-and-benchmarks/)

## Builds on (most cited first, 30 of 32)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation](https://mlsanity.com/paper/2308.08155/) (2023)
- [Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters](https://mlsanity.com/paper/2408.03314/) (2024)
- [Large Language Monkeys: Scaling Inference Compute with Repeated Sampling](https://mlsanity.com/paper/2407.21787/) (2024)
- [BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agents](https://mlsanity.com/paper/2504.12516/) (2025)
- [A Survey on the Memory Mechanism of Large Language Model based Agents](https://mlsanity.com/paper/2404.13501/) (2024)
- [Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMs](https://mlsanity.com/paper/2412.21187/) (2024)
- [Understanding the planning of LLM agents: A survey](https://mlsanity.com/paper/2402.02716/) (2024)
- [BERT Loses Patience: Fast and Robust Inference with Early Exit](https://mlsanity.com/paper/2006.04152/) (2020)
- [Token-Budget-Aware LLM Reasoning](https://mlsanity.com/paper/2412.18547/) (2024)
- [Cut the Crap: An Economical Communication Pipeline for LLM-based Multi-Agent Systems](https://mlsanity.com/paper/2410.02506/) (2024)
- [OWL: Optimized Workforce Learning for General Multi-Agent Assistance in Real-World Task Automation](https://mlsanity.com/paper/2505.23885/) (2025)
- [Agents: An Open-source Framework for Autonomous Language Agents](https://mlsanity.com/paper/2309.07870/) (2023)
- [BERT-of-Theseus: Compressing BERT by Progressive Module Replacing](https://mlsanity.com/paper/2002.02925/) (2020)
- [Symbolic Learning Enables Self-Evolving Agents](https://mlsanity.com/paper/2406.18532/) (2024)
- [Scaling Test-time Compute for LLM Agents](https://mlsanity.com/paper/2506.12928/) (2025)
- [BERT Learns to Teach: Knowledge Distillation with Meta Learning](https://mlsanity.com/paper/2106.04570/) (2021)
- [EfficientVLM: Fast and Accurate Vision-Language Models via Knowledge Distillation and Modal-adaptive Pruning](https://mlsanity.com/paper/2210.07795/) (2022)
- [Reasoning in Token Economies: Budget-Aware Evaluation of LLM Reasoning Strategies](https://mlsanity.com/paper/2406.06461/) (2024)
- [Beyond Preserved Accuracy: Evaluating Loyalty and Robustness of BERT Compression](https://mlsanity.com/paper/2109.03228/) (2021)
- [Cost-of-Pass: An Economic Framework for Evaluating Language Models](https://mlsanity.com/paper/2504.13359/) (2025)
- [BudgetMLAgent: A Cost-Effective LLM Multi-Agent system for Automating Machine Learning Tasks](https://mlsanity.com/paper/2411.07464/) (2024)
- [Efficient Prompting via Dynamic In-Context Learning](https://mlsanity.com/paper/2305.11170/) (2023)

## Built on by (most cited first, 30 of 32)

- [ToolOrchestra: Elevating Intelligence via Efficient Model and Tool Orchestration](https://mlsanity.com/paper/2511.21689/) (2025)
- [Budget-Aware Tool Use Enables Effective Agent Scaling](https://mlsanity.com/paper/2511.17006/) (2025)
- [How Do AI Agents Spend Your Money? Analyzing and Predicting Token Consumption in Agentic Coding Tasks](https://mlsanity.com/paper/2604.22750/) (2026)
- [Flash-Searcher: Fast and Effective Web Agents via DAG-Based Parallel Execution](https://mlsanity.com/paper/2509.25301/) (2025)
- [Towards Personalized Deep Research: Benchmarks and Evaluations](https://mlsanity.com/paper/2509.25106/) (2025)
- [Toward Efficient Agents: Memory, Tool learning, and Planning](https://mlsanity.com/paper/2601.14192/) (2026)
- [Inference-Time Budget Control for LLM Search Agents](https://mlsanity.com/paper/2605.05701/) (2026)
- [Stop Wasting Your Tokens: Towards Efficient Runtime Multi-Agent Systems](https://mlsanity.com/paper/2510.26585/) (2025)
- [How Far Are We from Genuinely Useful Deep Research Agents?](https://mlsanity.com/paper/2512.01948/) (2025)
- [TPS-Bench: Evaluating AI Agents' Tool Planning & Scheduling Abilities in Compounding Tasks](https://mlsanity.com/paper/2511.01527/) (2025)
- [AtlasVA: Self-Evolving Visual Skill Memory for Teacher-Free VLM Agents](https://mlsanity.com/paper/2605.17933/) (2026)
- [Why Search When You Can Transfer? Amortized Agentic Workflow Design from Structural Priors](https://mlsanity.com/paper/2604.25012/) (2026)
- [Agent-Omit: Adaptive Context Omission for Efficient LLM Agents](https://mlsanity.com/paper/2602.04284/) (2026)
- [PreFlect: From Retrospective to Prospective Reflection in Large Language Model Agents](https://mlsanity.com/paper/2602.07187/) (2026)
- [Beyond Accuracy: Unveiling Inefficiency Patterns in Tool-Integrated Reasoning](https://mlsanity.com/paper/2604.05404/) (2026)
- [Efficient Agent Evaluation via Diversity-Guided User Simulation](https://mlsanity.com/paper/2604.21480/) (2026)
- [Only Ask What You Don't Know: Grounded Delta Planning for Efficient Multi-step RAG](https://mlsanity.com/paper/2606.22681/) (2026)
- [When Agents Coordinate: Measuring Coordination in Multi-Agent AI Coding](https://mlsanity.com/paper/2608.16801/) (2026)
- [Reliable agent engineering should integrate machine-compatible organizational principles](https://mlsanity.com/paper/2512.07665/) (2025)
- [Cost and Accuracy of Long-Term Memory in Distributed Multi-Agent Systems Based on Large Language Models](https://mlsanity.com/paper/2601.07978/) (2026)
- [Last Step Matters: Early Uncertainty Cannot Predict Failure in Long-Horizon Agents](https://mlsanity.com/paper/2608.29685/) (2026)
- [CAM: A Causality-based Analysis Framework for Multi-Agent Code Generation Systems](https://mlsanity.com/paper/2602.02138/) (2026)
- [AgentSpec: Speculative Decoding for Batch Inference of LLM Agents](https://mlsanity.com/paper/2608.24004/) (2026)
- [The Missing Piece in Pre-trained Model Evaluation: Reward-Guided Decoding Unlocks Task-Oriented Behavior Without Parameter Updates](https://mlsanity.com/paper/2605.28020/) (2026)
- [Can Agentic Trading Systems Pay for Their Own Intelligence?](https://mlsanity.com/paper/2607.10286/) (2026)
- [The Hidden Footprint: Making Storage a First-Class Metric for LLM Agent Evaluation](https://mlsanity.com/paper/2607.11149/) (2026)
- [OpenCodeReview: Determinism over Non-Determinism for Cost-Effective Agent-Based Code Review](https://mlsanity.com/paper/2608.09290/) (2026)
- [Don't Overthink, Don't Underthink: Toward Adaptive Reasoning in Agentic AI](https://mlsanity.com/paper/2608.26442/) (2026)
- [Defining AI Agents: A Compendium of Criteria, Metrics, and Benchmarks](https://mlsanity.com/paper/2609.11018/) (2026)
- [LIMBO: Lifelong Inference-Time Memory and Budget Optimization for LLM Agents](https://mlsanity.com/paper/2609.14138/) (2026)

## Abstract (arXiv)

The remarkable capabilities of Large Language Model (LLM)-driven agents have enabled sophisticated systems to tackle complex, multi-step tasks, but their escalating costs threaten scalability and accessibility. This work presents the first systematic study of the efficiency-effectiveness trade-off in modern agent systems, addressing the critical need for cost-effective designs without sacrificing performance. We investigate three key questions: (1) How much complexity do agentic tasks inherently require? (2) When do additional modules yield diminishing returns? (3) How much efficiency can be gained through the design of efficient agent frameworks? Through an empirical analysis on the GAIA benchmark, we evaluate the impact of LLM backbone selection, agent framework designs, and test-time scaling strategies. Using the cost-of-pass metric, we quantify the efficiency-performance trade-off across these dimensions. Our findings inform the development of Efficient Agents , a novel agent framework that has an optimal complexity to task requirements. Efficient Agents retains 96.7% of the performance of OWL, one leading open-source agent framework, while reducing operational costs from 0.398 to 0.228, resulting in a 28.4% improvement in cost-of-pass. Our work provides actionable insights for designing efficient, high-performing agent systems, advancing the accessibility and sustainability of AI-driven solutions.