mlsanityData to 7 Oct 2026

BudgetMLAgent: A Cost-Effective LLM Multi-Agent system for Automating Machine Learning Tasks

arXiv 2411.07464 ↗First posted 12 Nov 2024cs.MA cs.AI cs.CLcited by 19

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

Large Language Models (LLMs) excel in diverse applications including generation of code snippets, but often struggle with generating code for complex Machine Learning (ML) tasks. Although existing LLM single-agent based systems give varying performance depending on the task complexity, they purely rely on larger and expensive models such as GPT-4. Our investigation reveals that no-cost and low-cost models such as Gemini-Pro, Mixtral and CodeLlama perform far worse than GPT-4 in a single-agent setting. With the motivation of developing a cost-efficient LLM based solution for solving ML tasks, we propose an LLM Multi-Agent based system which leverages combination of experts using profiling, efficient retrieval of past observations, LLM cascades, and ask-the-expert calls. Through empirical analysis on ML engineering tasks in the MLAgentBench benchmark, we demonstrate the effectiveness of our system, using no-cost models, namely Gemini as the base LLM, paired with GPT-4 in cascade and expert to serve occasional ask-the-expert calls for planning. With 94.2% reduction in the cost (from 0.931 per run cost averaged over all tasks for GPT-4 single agent system to 0.054), our system is able to yield better average success rate of 32.95% as compared to GPT-4 single-agent system yielding 22.72% success rate averaged over all the tasks of MLAgentBench.

Key facts. Cited by 19 papers in the map (data through 7 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 22 of its 46 references that are in the map. First posted on arXiv 12 Nov 2024.

Builds on

22 of 46 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Small LLMs Are Weak Tool Learners: A Multi-LLM Agent14 Jan

Other references · 21

  1. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  2. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
  3. 2020Open Graph Benchmark: Datasets for Machine Learning on Graphscited by 2,148
  4. 2023Code Llama: Open Foundation Models for Codecited by 2,105
  5. 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,906
  6. 2023FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performancecited by 791
  7. 2022Towards Reasoning in Large Language Models: A Surveycited by 593
  8. 2024Executable Code Actions Elicit Better LLM Agentscited by 537
  9. 2019AutoML: A Survey of the State-of-the-Artcited by 388
  10. 2024Hybrid LLM: Cost-Efficient and Quality-Aware Query Routingcited by 354
  11. 2024CodeAgent: Enhancing Code Generation with Tool-Integrated Agent Systems for Real-World Repo-level Coding Challengescited by 278
  12. 2024A Survey on Recent Advances in LLM-Based Multi-turn Dialogue Systemscited by 170
9 more
  1. 2024Large Language Models(LLMs) on Tabular Data: Prediction, Generation, and Understanding -- A Surveycited by 157
  2. 2023Large Language Model Cascades with Mixture of Thoughts Representations for Cost-efficient Reasoningcited by 140
  3. 2022Large Language Models Meet NL2Code: A Surveycited by 135
  4. 2024Debug like a Human: A Large Language Model Debugger via Verifying Runtime Execution Step-by-stepcited by 123
  5. 2023MLCopilot: Unleashing the Power of Large Language Models in Solving Machine Learning Taskscited by 66
  6. 2023ML-Bench: Evaluating Large Language Models and Agents for Machine Learning Tasks on Repository-Level Codecited by 63
  7. 2023Call for Papers -- The BabyLM Challenge: Sample-efficient pretraining on a developmentally plausible corpuscited by 62
  8. 2023Unifying the Perspectives of NLP and Software Engineering: A Survey on Language Models for Codecited by 62
  9. 2024A comparison of Human, GPT-3.5, and GPT-4 Performance in a University-Level Coding Coursecited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 12; 2026: 7.

19 papers.

Sort:

By area:

  1. 2025Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systemscited by 125
  2. 2025From Automation to Autonomy: A Survey on Large Language Models in Scientific Discoverycited by 86
  3. 2025Efficient Agents: Building Effective Agents While Reducing Costcited by 32
  4. 2025CostBench: Evaluating Multi-Turn Cost-Optimal Planning and Adaptation in Dynamic Environments for LLM Tool-Use Agentscited by 32
  5. 2025Large Language Model-based Data Science Agent: A Surveycited by 22
  6. 2025Agent.xpu: Efficient Scheduling of Agentic LLM Workloads on Heterogeneous SoCcited by 11
  7. 2026Toward Efficient Agents: Memory, Tool learning, and Planningcited by 11
  8. 2025More with Less: An Empirical Study of Turn-Control Strategies for Efficient Coding Agentscited by 8
  9. 2025SimuGen: Multi-modal Agentic Framework for Constructing Block Diagram-Based Simulation Modelscited by 7
  10. 2026EET: Experience-Driven Early Termination for Cost-Efficient Software Engineering Agentscited by 6
  11. 2025Optimizing PyTorch Inference with LLM-Based Multi-Agent Systemscited by 4
  12. 2025ASAP: an Agentic Solution to Auto-optimize Performance of Large-Scale LLM Trainingcited by 2
7 more
  1. 2025Tool-RoCo: An Agent-as-Tool Self-organization Large Language Model Benchmark in Multi-robot Cooperationcited by 2
  2. 2026Cost and Accuracy of Long-Term Memory in Distributed Multi-Agent Systems Based on Large Language Modelscited by 2
  3. 2026LLM-Based Multi-Agent Systems for Code Generation: A Multi-Vocal Literature Reviewcited by 2
  4. 2026Can AI Models Direct Each Other? Organizational Structure as a Probe into Training Limitationscited by 1
  5. 2025MetaLint: Easy-to-Hard Generalization for Code Lintingcited by 0
  6. 2026Trustworthy Self-Composable Big-Data-as-a-Service: An LLM-Orchestrated Multi-Agent Framework for Automated Data Engineering, AutoML, MLOps Deployment, and Drift-Aware Lifecycle Optimizationcited by 0
  7. 2026What Can Component-Replacement Evidence Establish? A Critical Scoping Review of Local Decisions in LLM Agentscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023TrainerAgent: Customizable and Efficient Model Training through LLM-Powered Multi-Agent System
  2. 2024Enhancing the General Agent Capabilities of Low-Parameter LLMs through Tuning and Multi-Branch Reasoning
  3. 2025MAS-GPT: Training LLMs to Build LLM-based Multi-Agent Systems
  4. 2025MLZero: A Multi-Agent System for End-to-end Machine Learning Automation
  5. 2024AutoML-Agent: A Multi-Agent LLM Framework for Full-Pipeline AutoML
  6. 2024Large Language Model Evaluation Via Multi AI Agents: Preliminary results
  7. 2025ML-Agent: Reinforcing LLM Agents for Autonomous Machine Learning Engineering
  8. 2024Offline Training of Language Model Agents with Functions as Learnable Weights