mlsanityData to 7 Oct 2026

FireAct: Toward Language Agent Fine-tuning

arXiv 2310.05915 ↗First posted 9 Oct 2023cs.CL cs.AI cs.LGcited by 247

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

Recent efforts have augmented language models (LMs) with external tools or environments, leading to the development of language agents that can reason and act. However, most of these agents rely on few-shot prompting techniques with off-the-shelf LMs. In this paper, we investigate and argue for the overlooked direction of fine-tuning LMs to obtain language agents. Using a setup of question answering (QA) with a Google search API, we explore a variety of base LMs, prompting methods, fine-tuning data, and QA tasks, and find language agents are consistently improved after fine-tuning their backbone LMs. For example, fine-tuning Llama2-7B with 500 agent trajectories generated by GPT-4 leads to a 77% HotpotQA performance increase. Furthermore, we propose FireAct, a novel approach to fine-tuning LMs with trajectories from multiple tasks and prompting methods, and show having more diverse fine-tuning data can further improve agents. Along with other findings regarding scaling effects, robustness, generalization, efficiency and cost, our work establishes comprehensive benefits of fine-tuning LMs for agents, and provides an initial set of experimental designs, insights, as well as open questions toward language agent fine-tuning.

Key facts. Cited by 247 papers in the map (data through 7 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 67 of its 87 references that are in the map. First posted on arXiv 9 Oct 2023.

Builds on

67 of 87 references.

Closest ancestors · 4

Same direction, newest first

  1. 2023The Rise and Potential of Large Language Model Based Agents: A Survey14 Sep
  2. 2023Cognitive Architectures for Language Agents5 Sep
  3. 2023A Survey on Large Language Model based Autonomous Agents22 Aug
  4. 2023SwiftSage: A Generative Agent with Fast and Slow Thinking for Complex Interactive Tasks27 May

Other references · 63

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  5. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  6. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  7. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  8. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  9. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  10. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  11. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  12. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
38 more
  1. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  2. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  3. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  4. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
  5. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
  6. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  7. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,855
  8. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  9. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,591
  10. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
  11. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
  12. 2023Code Llama: Open Foundation Models for Codecited by 2,105
  13. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
  14. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,037
  15. 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 1,908
  16. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,880
  17. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,722
  18. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,462
  19. 2023Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understandingcited by 1,461
  20. 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,302
  21. 2023Mind2Web: Towards a Generalist Agent for the Webcited by 1,210
  22. 2022WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agentscited by 1,197
  23. 2022Code as Policies: Language Model Programs for Embodied Controlcited by 1,172
  24. 2023LIMA: Less Is More for Alignmentcited by 1,080
  25. 2022Measuring and Narrowing the Compositionality Gap in Language Modelscited by 1,054
  26. 2022Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agentscited by 1,029
  27. 2022STaR: Bootstrapping Reasoning With Reasoningcited by 895
  28. 2021Show Your Work: Scratchpads for Intermediate Computation with Language Modelscited by 871
  29. 2019DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generationcited by 842
  30. 2023StarCoder: may the source be with you!cited by 790
  31. 2021Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategiescited by 736
  32. 2023The Flan Collection: Designing Data and Methods for Effective Instruction Tuningcited by 734
  33. 2023WizardCoder: Empowering Code Large Language Models with Evol-Instructcited by 728
  34. 2023OpenAssistant Conversations -- Democratizing Large Language Model Alignmentcited by 712
  35. 2021Cross-Task Generalization via Natural Language Crowdsourcing Instructionscited by 696
  36. 2022Large Language Models Can Self-Improvecited by 656
  37. 2023Instruction Tuning for Large Language Models: A Surveycited by 617
  38. 2023LLM+P: Empowering Large Language Models with Optimal Planning Proficiencycited by 496

50 most cited of 63 All 63 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 7; 2024: 64; 2025: 84; 2026: 92.

247 papers.

Sort:

By area:

  1. 2023CogAgent: A Visual Language Model for GUI Agentscited by 692
  2. 2024Executable Code Actions Elicit Better LLM Agentscited by 537
  3. 2025ToolRL: Reward is All Tool Learning Needscited by 342
  4. 2024Training Software Engineering Agents and Verifiers with SWE-Gymcited by 300
  5. 2024Quiet-STaR: Language Models Can Teach Themselves to Think Before Speakingcited by 292
  6. 2025Memory in the Age of AI Agentscited by 285
  7. 2024A Survey on Knowledge Distillation of Large Language Modelscited by 276
  8. 2024Personal LLM Agents: Insights and Survey about the Capability, Efficiency and Securitycited by 257
  9. 2025SWE-smith: Scaling Data for Software Engineering Agentscited by 255
  10. 2024WebRL: Training LLM Web Agents via Self-Evolving Online Curriculum Reinforcement Learningcited by 243
  11. 2025WebSailor: Navigating Super-human Reasoning for Web Agentcited by 218
  12. 2024ArCHer: Training Language Model Agents via Hierarchical Multi-Turn RLcited by 212
38 more
  1. 2024DigiRL: Training In-The-Wild Device-Control Agents with Autonomous Reinforcement Learningcited by 210
  2. 2025The Landscape of Agentic Reinforcement Learning for LLMs: A Surveycited by 195
  3. 2024Trial and Error: Exploration-Based Trajectory Optimization for LLM Agentscited by 194
  4. 2024APIGen: Automated Pipeline for Generating Verifiable and Diverse Function-Calling Datasetscited by 188
  5. 2025WebDancer: Towards Autonomous Information Seeking Agencycited by 180
  6. 2024StableToolBench: Towards Stable Large-Scale Benchmarking on Tool Learning of Large Language Modelscited by 129
  7. 2025OWL: Optimized Workforce Learning for General Multi-Agent Assistance in Real-World Task Automationcited by 129
  8. 2024AgentGym: Evolving Large Language Model-based Agents across Diverse Environmentscited by 120
  9. 2026SoK: Agentic Skills -- Beyond Tool Use in LLM Agentscited by 120
  10. 2024DS-Agent: Automated Data Science by Empowering Large Language Models with Case-Based Reasoningcited by 119
  11. 2024xLAM: A Family of Large Action Models to Empower AI Agent Systemscited by 108
  12. 2025A Survey of Frontiers in LLM Reasoning: Inference Scaling, Learning to Reason, and Agentic Systemscited by 108
  13. 2024A Survey on Large Language Model-Based Game Agentscited by 106
  14. 2025Reinforcement Learning for Long-Horizon Interactive LLM Agentscited by 103
  15. 2024Agent-FLAN: Designing Data and Methods of Effective Agent Tuning for Large Language Modelscited by 97
  16. 2024AutoGLM: Autonomous Foundation Agents for GUIscited by 96
  17. 2024Keeping LLMs Aligned After Fine-tuning: The Crucial Role of Prompt Templatescited by 94
  18. 2025Agent Lightning: Train ANY AI Agents with Reinforcement Learningcited by 94
  19. 2025Learn-by-interact: A Data-Centric Framework for Self-Adaptive Agents in Realistic Environmentscited by 93
  20. 2024Small LLMs Are Weak Tool Learners: A Multi-LLM Agentcited by 92
  21. 2025Reinforcement Learning for Self-Improving Agent with Skill Librarycited by 91
  22. 2024Watch Every Step! LLM Agent Learning via Iterative Step-Level Process Refinementcited by 86
  23. 2026OpenClaw-RL: Train Any Agent Simply by Talkingcited by 83
  24. 2025PlanGenLLMs: A Modern Survey of LLM Planning Capabilitiescited by 82
  25. 2024AutoAct: Automatic Agent Learning from Scratch for QA via Self-Planningcited by 81
  26. 2024AgentGen: Enhancing Planning Abilities for Large Language Model based Agent via Environment and Task Generationcited by 80
  27. 2025Lifelong Learning of Large Language Model based Agents: A Roadmapcited by 78
  28. 2023LLMLight: Large Language Models as Traffic Signal Control Agentscited by 73
  29. 2023Agent Lumos: Unified and Modular Training for Open-Source Language Agentscited by 72
  30. 2023Igniting Language Intelligence: The Hitchhiker's Guide From Chain-of-Thought Reasoning to Language Agentscited by 72
  31. 2024ToolGen: Unified Tool Retrieval and Calling via Generationcited by 72
  32. 2024Policy Agnostic RL: Offline RL and Online RL Fine-Tuning of Any Class and Backbonecited by 71
  33. 2024AvaTaR: Optimizing LLM Agents for Tool Usage via Contrastive Reasoningcited by 69
  34. 2025AgentPRM: Process Reward Models for LLM Agents via Step-Wise Promise and Progresscited by 69
  35. 2025SPA-RL: Reinforcing LLM Agents via Stepwise Progress Attributioncited by 68
  36. 2025AgentGym-RL: Training LLM Agents for Long-Horizon Decision Making through Multi-Turn Reinforcement Learningcited by 68
  37. 2024Agent Planning with World Knowledge Modelcited by 66
  38. 2024Direct Multi-Turn Preference Optimization for Language Agentscited by 63

50 most cited of 247 All 247 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Enhancing the General Agent Capabilities of Low-Parameter LLMs through Tuning and Multi-Branch Reasoning
  2. 2025Self-Improving LLM Agents at Test-Time
  3. 2026Search, Do not Guess: Teaching Small Language Models to Be Effective Search Agents
  4. 2026FT-Dojo: Towards Autonomous LLM Fine-Tuning with Language Agents
  5. 2024The Ultimate Guide to Fine-Tuning LLMs from Basics to Breakthroughs: An Exhaustive Review of Technologies, Research, Best Practices, Applied Research Challenges and Opportunities
  6. 2023AgentTuning: Enabling Generalized Agent Abilities for LLMs
  7. 2023Language Agent Tree Search Unifies Reasoning Acting and Planning in Language Models
  8. 2024CMAT: A Multi-Agent Collaboration Tuning Framework for Enhancing Small Language Models