mlsanityData to 7 Oct 2026

EcoAssistant: Using LLM Assistant More Affordably and Accurately

arXiv 2310.03046 ↗First posted 3 Oct 2023cs.SE cs.AIcited by 47

Not on the map. Why?

Abstract (arXiv)

Today, users ask Large language models (LLMs) as assistants to answer queries that require external knowledge; they ask about the weather in a specific city, about stock prices, and even about where specific locations are within their neighborhood. These queries require the LLM to produce code that invokes external APIs to answer the user's question, yet LLMs rarely produce correct code on the first try, requiring iterative code refinement upon execution results. In addition, using LLM assistants to support high query volumes can be expensive. In this work, we contribute a framework, EcoAssistant, that enables LLMs to answer code-driven queries more affordably and accurately. EcoAssistant contains three components. First, it allows the LLM assistants to converse with an automatic code executor to iteratively refine code or to produce answers based on the execution results. Second, we use a hierarchy of LLM assistants, which attempts to answer the query with weaker, cheaper LLMs before backing off to stronger, expensive ones. Third, we retrieve solutions from past successful queries as in-context demonstrations to help subsequent queries. Empirically, we show that EcoAssistant offers distinct advantages for affordability and accuracy, surpassing GPT-4 by 10 points of success rate with less than 50% of GPT-4's cost.

Key facts. Cited by 47 papers in the map (data through 7 Oct 2026). Builds on 25 of its 30 references that are in the map. First posted on arXiv 3 Oct 2023.

Builds on

25 of 30 references.

References · 25

  1. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  3. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  4. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  5. 2021Program Synthesis with Large Language Modelscited by 3,509
  6. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,203
  7. 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Frameworkcited by 1,761
  8. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,739
  9. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,722
  10. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,422
  11. 2023AgentBench: Evaluating LLMs as Agentscited by 1,211
  12. 2023Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debatecited by 995
13 more
  1. 2021Measuring Coding Challenge Competence With APPScited by 879
  2. 2023FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performancecited by 791
  3. 2017Reading Wikipedia to Answer Open-Domain Questionscited by 777
  4. 2021CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generationcited by 732
  5. 2019Latent Retrieval for Weakly Supervised Open Domain Question Answeringcited by 691
  6. 2023Is ChatGPT a Good NLG Evaluator? A Preliminary Studycited by 420
  7. 2023Learning to Compress Prompts with Gist Tokenscited by 379
  8. 2023GPTScore: Evaluate as You Desirecited by 357
  9. 2023Multi-Agent Collaboration: Harnessing the Power of Intelligent LLM Agentscited by 329
  10. 2023InterCode: Standardizing and Benchmarking Interactive Coding with Execution Feedbackcited by 229
  11. 2023Cost-Effective Hyperparameter Optimization for Large Language Model Generation Inferencecited by 40
  12. 2023LeanContext: Cost-Efficient Domain-Specific Question Answering Using LLMscited by 40
  13. 2023BatchPrompt: Accomplish more with lesscited by 16

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 18; 2025: 13; 2026: 13.

47 papers.

Sort:

By area:

  1. 2024Visual Sketchpad: Sketching as a Visual Chain of Thought for Multimodal Language Modelscited by 260
  2. 2025Which Agent Causes Task Failures and When? On Automated Failure Attribution of LLM Multi-Agent Systemscited by 170
  3. 2023Pearl: Personalizing Large Language Model Writing Assistants with Generation-Calibrated Retrieverscited by 91
  4. 2025Agent KB: Leveraging Cross-Domain Experience for Agentic Problem Solvingcited by 90
  5. 2025Harnessing Multiple Large Language Models: A Survey on LLM Ensemblecited by 86
  6. 2024EHRAgent: Code Empowers Large Language Models for Few-shot Complex Tabular Reasoning on Electronic Health Recordscited by 75
  7. 2024Learning to Route LLMs with Confidence Tokenscited by 66
  8. 2024Model Compression and Efficient Inference for Large Language Models: A Surveycited by 62
  9. 2025RouterEval: A Comprehensive Benchmark for Routing LLMs to Explore Model-level Scaling Up in LLMscited by 58
  10. 2025A Survey of LLM-Driven AI Agent Communication: Protocols, Security Risks, and Defense Countermeasurescited by 54
  11. 2024m&m's: A Benchmark to Evaluate Tool-Use for multi-step multi-modal Taskscited by 38
  12. 2024Adaptive In-conversation Team Building for Language Model Agentscited by 38
35 more
  1. 2024Agent Design Pattern Catalogue: A Collection of Architectural Patterns for Foundation Model based Agentscited by 35
  2. 2025CostBench: Evaluating Multi-Turn Cost-Optimal Planning and Adaptation in Dynamic Environments for LLM Tool-Use Agentscited by 32
  3. 2024Are More LLM Calls All You Need? Towards Scaling Laws of Compound Inference Systemscited by 24
  4. 2024Large Language Models for Manufacturingcited by 24
  5. 2024Multi-Layer Transformers Gradient Can be Approximated in Almost Linear Timecited by 22
  6. 2025Doing More with Less: A Survey on Routing Strategies for Resource Optimisation in Large Language Model-Based Systemscited by 18
  7. 2025Automatic Failure Attribution and Critical Step Prediction Method for Multi-Agent Systems Based on Causal Inferencecited by 18
  8. 2024Interactive Speculative Planning: Enhance Agent Efficiency through Co-design of System and User Interfacecited by 15
  9. 2024CigaR: Cost-efficient Program Repair with LLMscited by 14
  10. 2026Inference-Time Budget Control for LLM Search Agentscited by 13
  11. 2023Advancing the Search Frontier with AI Agentscited by 12
  12. 2024EcoAct: Economic Agent Determines When to Register What Actioncited by 11
  13. 2026The Evolution of Tool Use in LLM Agents: From Single-Tool Call to Multi-Tool Orchestrationcited by 11
  14. 2025Dynamic Speculative Agent Planningcited by 10
  15. 2026Spend Less, Reason Better: Budget-Aware Value Tree Search for LLM Agentscited by 10
  16. 2024Enabling Weak LLMs to Judge Response Reliability via Meta Rankingcited by 9
  17. 2025Orchestrating Agents and Data for Enterprise: A Blueprint Architecture for Compound AIcited by 9
  18. 2024Affordable Generative Agentscited by 8
  19. 2024CodeNav: Beyond tool-use to using real-world codebases with LLM agentscited by 8
  20. 2026Task-Aware LLM Routing with Multi-Level Task-Profile-Guided Data Synthesis for Cold-Start Scenarioscited by 8
  21. 2026DualSpec: Accelerating Deep Research Agents via Dual-Process Action Speculationcited by 6
  22. 2025Towards Efficient Multi-LLM Inference: Characterization and Analysis of LLM Routing and Hierarchical Techniquescited by 5
  23. 2025Neural Algorithmic Reasoning for Hypergraphs with Looped Transformerscited by 3
  24. 2026From Sampled Outcomes to Capability Distributions: Rethinking Supervision for LLM Routingcited by 3
  25. 2026RAL2M: Retrieval Augmented Learning-To-Match Against Hallucination in Compliance-Guaranteed Service Systemscited by 2
  26. 2026Tandem: Riding Together with Large and Small Language Models for Efficient Reasoningcited by 2
  27. 2024Trigger3: Refining Query Correction via Adaptive Model Selectorcited by 1
  28. 2025An Empirical Study on Strong-Weak Model Collaboration for Repo-level Code Generationcited by 1
  29. 2026Computational Arbitrage in AI Model Marketscited by 1
  30. 2023Redefining Developer Assistance: Through Large Language Models in Software Ecosystemcited by 0
  31. 2024XTRUST: On the Multilingual Trustworthiness of Large Language Modelscited by 0
  32. 2026CROP: Token-Efficient Reasoning in Large Language Models via Regularized Prompt Optimizationcited by 0
  33. 2026HyDRA: Hybrid Dynamic Routing Architecture for Heterogeneous LLM Poolscited by 0
  34. 2026When Efficiency Backfires: Cascading LLMs Trigger Cascade Failure under Adversarial Attackcited by 0
  35. 2026When to Call an LLM: A Confidence-Gated Hybrid for Cost-Effective Emotion Recognition in Conversational AIcited by 0