mlsanityData to 7 Oct 2026

Offline Training of Language Model Agents with Functions as Learnable Weights

arXiv 2402.11359 ↗First posted 17 Feb 2024cs.AI cs.CLcited by 47

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

Researchers and practitioners have recently reframed powerful Large Language Models (LLMs) as agents, enabling them to automate complex tasks largely via the use of specialized functions. To facilitate the development of LLM agents, we present a novel paradigm of training LLM agents without modifying the LLM weights, which is particularly useful when the LLMs are difficult or inaccessible for modifications. Inspired by how humans continuously forge tools to adapt to real-world tasks, rather than change our biological structure to fit a static set of tools, we propose to progressively forge agent's functions to better solve the downstream tasks instead of modifying the LLM weights. By treating the functions as learnable `agent parameters' and leveraging the fundamental idea of model training in artificial intelligence, we develop AgentOptimizer that employs the LLM to update agents' functions and devise an agent training algorithm with two strategies, roll-back, and early-stop, to streamline the training process. With extensive experiments, we showcase that the agent training paradigm could significantly improve the performance of representative LLM agents in various downstream tasks. We also study the behavior of the agent training regarding aspects like the learning curve and domain transferability.

Key facts. Cited by 47 papers in the map (data through 7 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 33 of its 46 references that are in the map. First posted on arXiv 17 Feb 2024.

Builds on

33 of 46 references.

Closest ancestors · 5

Same direction, newest first

  1. 2023AgentTuning: Enabling Generalized Agent Abilities for LLMs19 Oct
  2. 2023The Rise and Potential of Large Language Model Based Agents: A Survey14 Sep
  3. 2023A Survey on Large Language Model based Autonomous Agents22 Aug
  4. 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework1 Aug
  5. 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society31 Mar

Other references · 28

  1. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  3. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  4. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  5. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
  6. 2023Mistral 7Bcited by 3,065
  7. 2023Code Llama: Open Foundation Models for Codecited by 2,105
  8. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
  9. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,880
  10. 2024Mixtral of Expertscited by 1,785
  11. 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,302
  12. 2020ALFWorld: Aligning Text and Embodied Environments for Interactive Learningcited by 1,143
16 more
  1. 2023GAIA: a benchmark for General AI Assistantscited by 1,084
  2. 2016An overview of gradient descent optimization algorithmscited by 1,021
  3. 2023Large Language Models as Optimizerscited by 830
  4. 2021Domain Generalization: A Surveycited by 760
  5. 2016Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimizationcited by 688
  6. 2023Check Your Facts and Try Again: Improving Large Language Models with External Knowledge and Automated Feedbackcited by 351
  7. 2022Dynamic Prompt Learning via Policy Gradient for Semi-structured Mathematical Reasoningcited by 347
  8. 2023Tool Learning with Foundation Modelscited by 302
  9. 2023Large Language Models as Tool Makerscited by 293
  10. 2023Large Language Models for Robotics: A Surveycited by 132
  11. 2023CRAFT: Customizing LLMs by Creating and Retrieving from Specialized Toolsetscited by 122
  12. 2023Evil Geniuses: Delving into the Safety of LLM-based Agentscited by 103
  13. 2018Revisiting Small Batch Training for Deep Neural Networkscited by 100
  14. 2023CREATOR: Tool Creation for Disentangling Abstract and Concrete Reasoning of Large Language Modelscited by 90
  15. 2022A data-driven approach for learning to control computerscited by 87
  16. 2023Using Large Language Models for Hyperparameter Optimizationcited by 71

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 9; 2025: 19; 2026: 19.

47 papers.

Sort:

By area:

  1. 2024Automated Design of Agentic Systemscited by 360
  2. 2025Which Agent Causes Task Failures and When? On Automated Failure Attribution of LLM Multi-Agent Systemscited by 170
  3. 2025Multi-Agent Design: Optimizing Agents with Better Prompts and Topologiescited by 106
  4. 2024Symbolic Learning Enables Self-Evolving Agentscited by 104
  5. 2025BEST-Route: Adaptive LLM Routing with Test-Time Optimal Computecited by 63
  6. 2025FlowReasoner: Reinforcing Query-Level Meta-Agentscited by 56
  7. 2024Explaining Length Bias in LLM-Based Preference Evaluationscited by 48
  8. 2025A Survey on the Optimization of Large Language Model-based Agentscited by 45
  9. 2025CoAct-1: Computer-using Multi-Agent System with Coding Actionscited by 33
  10. 2025From Language to Action: A Review of Large Language Models as Autonomous Agents and Tool Userscited by 32
  11. 2024G"odel Agent: A Self-Referential Agent Framework for Recursive Self-Improvementcited by 27
  12. 2025Heterogeneous Swarms: Jointly Optimizing Model Roles and Weights for Multi-LLM Systemscited by 22
35 more
  1. 2025A Comprehensive Survey on Reinforcement Learning-based Agentic Search: Foundations, Roles, Optimizations, Evaluations, and Applicationscited by 21
  2. 2024REVOLVE: Optimizing AI Systems by Tracking Response Evolution in Textual Optimizationcited by 20
  3. 2026Flow caching for autoregressive video generationcited by 19
  4. 2025Automatic Failure Attribution and Critical Step Prediction Method for Multi-Agent Systems Based on Causal Inferencecited by 18
  5. 2025SwarmAgentic: Towards Fully Automated Agentic System Generation via Swarm Intelligencecited by 17
  6. 2026ProRL Agent: Rollout-as-a-Service for RL Training of Multi-Turn LLM Agentscited by 17
  7. 2025AgentTTS: Large Language Model Agent for Test-time Compute-optimal Scaling Strategy in Complex Taskscited by 16
  8. 2025A Survey on Mathematical Reasoning and Optimization with Large Language Modelscited by 12
  9. 2024EcoAct: Economic Agent Determines When to Register What Actioncited by 11
  10. 2026Who&When Pro: Can LLMs Really Attribute Failures in AI Agents?cited by 11
  11. 2026AutoSaddler: Automatic Harness Optimization with Durable Updates from Agent Execution Tracescited by 9
  12. 2025Memento No More: Coaching AI Agents to Master Multiple Tasks via Hints Internalizationcited by 8
  13. 2025POQD: Performance-Oriented Query Decomposer for Multi-vector retrievalcited by 5
  14. 2025A Survey on Collaborating Small and Large Language Models for Performance, Cost-effectiveness, Cloud-edge Privacy, and Trustworthinesscited by 5
  15. 2024Deploying Foundation Model Powered Agent Services: A Surveycited by 4
  16. 2024Library Learning Doesn't: The Curious Case of the Single-Use "Library"cited by 3
  17. 2025Divide, Optimize, Merge: Fine-Grained LLM Agent Optimization at Scalecited by 3
  18. 2025AgentBalance: Backbone-then-Topology Design for Cost-Effective Multi-Agent Systems under Budget Constraintscited by 3
  19. 2026Doc2Spec: Synthesizing Formal Programming Specifications from Natural Language via Grammar Inductioncited by 2
  20. 2026Learning How and What to Memorize: Cognition-Inspired Two-Stage Optimization for Evolving Memorycited by 2
  21. 2026Harness Continual Learning: Continual Adaptation Beyond Model Parameterscited by 2
  22. 2024Scaffolded Language Models with Language Supervision for Mixed-Autonomy: A Surveycited by 1
  23. 2025FutureWeaver: Planning Test-Time Compute for Multi-Agent Systems with Modularized Collaborationcited by 1
  24. 2026Secure Forgetting: A Framework for Privacy-Driven Unlearning in Large Language Model (LLM)-Based Agentscited by 1
  25. 2026FinEvo-Bench: A Longitudinal Benchmark for Self-Evolving Agents in Professional Financial Workflowscited by 1
  26. 2026MERA: Model Evolution and Routing with Skill Adaptation for Agentic Systems at Scalecited by 1
  27. 2026Story2Proposal: A Scaffold for Structured Scientific Paper Writingcited by 0
  28. 2026ALGOGEN: Tool-Generated Verifiable Traces for Reliable Algorithm Visualizationcited by 0
  29. 2026Eta Given Delta: Defining LLM Tool Efficiency With Marginal Tool Utilitycited by 0
  30. 2026EMAS: Stabilizing Multi-Agent System Evolution through Evidence-Guided Revisioncited by 0
  31. 2026SAGE: Self-Evolving Storyboard Skills via Attribution-Guided Rule Evolutioncited by 0
  32. 2026VideoResearcher: Self-Improving Tool Design for Long-Video Understandingcited by 0
  33. 2026PluginRSI: Recursive Improvement of Agent Harnesses with Reusable Pluginscited by 0
  34. 2026From Weak Task Specifications to Scientific Extraction Agents: Optimizing Task Constructioncited by 0
  35. 2026Harness Compilation: Which Decisions Should a Small Vision-Language Model Keep?cited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024AgentGym: Evolving Large Language Model-based Agents across Diverse Environments
  2. 2024Agent-FLAN: Designing Data and Methods of Effective Agent Tuning for Large Language Models
  3. 2025AgentRefine: Enhancing Agent Generalization through Refinement Tuning
  4. 2023TrainerAgent: Customizable and Efficient Model Training through LLM-Powered Multi-Agent System
  5. 2024Enhancing the General Agent Capabilities of Low-Parameter LLMs through Tuning and Multi-Branch Reasoning
  6. 2025Adaptation of Agentic AI: A Survey of Post-Training, Memory, and Skills
  7. 2025Scaling Agents via Continual Pre-training
  8. 2025Large Language Model Agent: A Survey on Methodology, Applications and Challenges