mlsanityData to 14 Oct 2026

Agent Lumos: Unified and Modular Training for Open-Source Language Agents

arXiv 2311.05657 ↗First posted 9 Nov 2023cs.AI cs.CL cs.LGcited by 72

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

Closed-source agents suffer from several issues such as a lack of affordability, transparency, and reproducibility, particularly on complex interactive tasks. This motivates the development of open-source alternatives. We introduce LUMOS, one of the first frameworks for training open-source LLM-based agents. LUMOS features a learnable, unified, and modular architecture with a planning module that learns high-level subgoal generation, and a grounding module trained to translate these into actions using various tools in the execution module. The design allows for modular upgrades and wider applicability to diverse interactive tasks. To foster generalizable agent learning, we collect large-scale, unified, and high-quality training annotations derived from diverse ground-truth reasoning rationales across various complex interactive tasks. On 9 datasets, LUMOS exhibits several key advantages: (1) LUMOS excels multiple larger open-source agents on the held-out datasets (unused for training) for each task type. LUMOS even surpasses GPT agents on QA and web tasks; (2) LUMOS outperforms open-source agents produced by chain-of-thoughts and unmodularized integrated training; and (3) LUMOS effectively generalizes to unseen tasks, outperforming 33B-scale agents and domain-specific agents.

Key facts. Cited by 72 papers in the map (data through 14 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 35 of its 49 references that are in the map. First posted on arXiv 9 Nov 2023.

Builds on

35 of 49 references.

Closest ancestors · 4

Same direction, newest first

  1. 2023AgentTuning: Enabling Generalized Agent Abilities for LLMs19 Oct
  2. 2023FireAct: Toward Language Agent Fine-tuning9 Oct
  3. 2023BOLAA: Benchmarking and Orchestrating LLM-augmented Autonomous Agents11 Aug
  4. 2023SwiftSage: A Generative Agent with Fast and Slow Thinking for Complex Interactive Tasks27 May

Other references · 31

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  3. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  4. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  5. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  6. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  7. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,824
  8. 2023Let's Verify Step by Stepcited by 3,789
  9. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
  10. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,955
  11. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,724
  12. 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 2,006
19 more
  1. 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answeringcited by 1,919
  2. 2022WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agentscited by 1,251
  3. 2023Mind2Web: Towards a Generalist Agent for the Webcited by 1,242
  4. 2020ALFWorld: Aligning Text and Embodied Environments for Interactive Learningcited by 1,205
  5. 2021Are NLP Models really able to Solve Simple Math Word Problems?cited by 878
  6. 2022Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Taskscited by 868
  7. 2017Crowdsourcing Multiple Choice Science Questionscited by 765
  8. 2021Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategiescited by 743
  9. 2023Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizescited by 737
  10. 2022A-OKVQA: A Benchmark for Visual Question Answering using World Knowledgecited by 710
  11. 2018From Recognition to Cognition: Visual Commonsense Reasoningcited by 556
  12. 2019COMET: Commonsense Transformers for Automatic Knowledge Graph Constructioncited by 450
  13. 2023How Far Can Camels Go? Exploring the State of Instruction Tuning on Open Resourcescited by 408
  14. 2021A Diverse Corpus for Evaluating and Developing English Math Word Problem Solverscited by 318
  15. 2021Symbolic Knowledge Distillation: from General Language Models to Commonsense Modelscited by 270
  16. 2023InterCode: Standardizing and Benchmarking Interactive Coding with Execution Feedbackcited by 238
  17. 2023ReWOO: Decoupling Reasoning from Observations for Efficient Augmented Language Modelscited by 184
  18. 2020Beyond I.I.D.: Three Levels of Generalization for Question Answering on Knowledge Basescited by 171
  19. 2018NL2Bash: A Corpus and Semantic Parser for Natural Language Interface to the Linux Operating Systemcited by 98

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 17; 2025: 35; 2026: 19.

72 papers.

Sort:

By area:

  1. 2024Aguvis: Unified Pure Vision Agents for Autonomous GUI Interactioncited by 278
  2. 2023The Unlocking Spell on Base LLMs: Rethinking Alignment via In-Context Learningcited by 257
  3. 2025WebAgent-R1: Training Web Agents via End-to-End Multi-Turn Reinforcement Learningcited by 165
  4. 2024AgentOccam: A Simple Yet Strong Baseline for LLM-Based Web Agentscited by 114
  5. 2025Learn-by-interact: A Data-Centric Framework for Self-Adaptive Agents in Realistic Environmentscited by 95
  6. 2025Lifelong Learning of Large Language Model based Agents: A Roadmapcited by 83
  7. 2024ToolGen: Unified Tool Retrieval and Calling via Generationcited by 74
  8. 2025InfiGUIAgent: A Multimodal Generalist GUI Agent with Native Reasoning and Reflectioncited by 67
  9. 2024Direct Multi-Turn Preference Optimization for Language Agentscited by 64
  10. 2024The Dawn of GUI Agent: A Preliminary Case Study with Claude 3.5 Computer Usecited by 63
  11. 2025AgentRefine: Enhancing Agent Generalization through Refinement Tuningcited by 53
  12. 2024SUPER: Evaluating Agents on Setting Up and Executing Tasks from Research Repositoriescited by 52
38 more
  1. 2026A Survey of Agentic Reasoning for Large Language Models: Towards Recursively Self-Improving and Collective Agentscited by 50
  2. 2025Distilling LLM Agent into Small Models with Retrieval and Code Toolscited by 47
  3. 2025A Survey on the Optimization of Large Language Model-based Agentscited by 46
  4. 2024MAGDi: Structured Distillation of Multi-Agent Interaction Graphs Improves Reasoning in Smaller Language Modelscited by 43
  5. 2025Large Language Models for Planning: A Comprehensive and Systematic Surveycited by 40
  6. 2024AgentCourt: Simulating Court with Adversarial Evolvable Lawyer Agentscited by 37
  7. 2024Benchmarking Agentic Workflow Generationcited by 37
  8. 2025Magnet: Multi-turn Tool-use Data Synthesis and Distillation via Graph Translationcited by 37
  9. 2025MemVerse: Multimodal Memory for Lifelong Learning Agentscited by 35
  10. 2025LLM-Powered GUI Agents in Phone Automation: Surveying Progress and Prospectscited by 32
  11. 2025From Language to Action: A Review of Large Language Models as Autonomous Agents and Tool Userscited by 32
  12. 2025AgentRM: Enhancing Agent Generalization with Reward Modelingcited by 27
  13. 2024AMOR: A Recipe for Building Adaptable Modular Knowledge Agents Through Process Feedbackcited by 25
  14. 2024Unlocking Reasoning Potential in Large Langauge Models by Scaling Code-form Planningcited by 25
  15. 2025Symbiotic Cooperation for Web Agents: Harnessing Complementary Strengths of Large and Small LLMscited by 24
  16. 2024ReAct Meets ActRe: When Language Agents Enjoy Training Data Autonomycited by 23
  17. 2024Revealing the Barriers of Language Agents in Planningcited by 20
  18. 2025QLASS: Boosting Language Agent Inference via Q-Guided Stepwise Searchcited by 20
  19. 2025Agent-RewardBench: Towards a Unified Benchmark for Reward Modeling across Perception, Planning, and Safety in Real-World Multimodal Agentscited by 20
  20. 2025A Goal Without a Plan Is Just a Wish: Efficient and Effective Global Planner Training for Long-Horizon Agent Taskscited by 19
  21. 2025SpiritSight Agent: Advanced GUI Agent with One Lookcited by 18
  22. 2024Smurfs: Multi-Agent System using Context-Efficient DFSDT for Tool Planningcited by 13
  23. 2025Advancing Language Multi-Agent Learning with Credit Re-Assignment for Interactive Environment Generalizationcited by 11
  24. 2025ComfyUI-Copilot: An Intelligent Assistant for Automated Workflow Developmentcited by 11
  25. 2024Towards Collaborative Intelligence: Propagating Intentions and Reasoning for Multi-Agent Coordination with Large Language Modelscited by 10
  26. 2024Cognitive Kernel: An Open-source Agent System towards Generalist Autopilotscited by 10
  27. 2025AndroidGen: Building an Android Language Agent under Data Scarcitycited by 10
  28. 2026Learning How to Remember: A Meta-Cognitive Management Method for Structured and Transferable Agent Memorycited by 10
  29. 2025Unleashing Embodied Task Planning Ability in LLMs via Reinforcement Learningcited by 9
  30. 2025Why Do Open-Source LLMs Struggle with Data Analysis? A Systematic Empirical Studycited by 7
  31. 2025Hephaestus: Improving Fundamental Agent Capabilities of Large Language Models through Continual Pre-Trainingcited by 6
  32. 2025On Accelerating Edge AI: Optimizing Resource-Constrained Environmentscited by 5
  33. 2025Building LLM Agents by Incorporating Insights from Computer Systemscited by 5
  34. 2026Compiling Agentic Workflows into LLM Weights: Near-Frontier Quality at Two Orders of Magnitude Less Costcited by 5
  35. 2025Putting on the Thinking Hats: A Survey on Chain of Thought Fine-tuning from the Perspective of Human Reasoning Mechanismcited by 4
  36. 2025Fed-SE: Federated Self-Evolution for Privacy-Constrained Multi-Environment LLM Agentscited by 3
  37. 2026Supervised Fine-Tuning versus Reinforcement Learning: A Study of Post-Training Methods for Large Language Modelscited by 3
  38. 2025NAACL2025 Tutorial: Adaptation of Large Language Modelscited by 2

50 most cited of 72 All 72 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023Enhancing Open-Domain Task-Solving Capability of LLMs via Autonomous Tool Integration from GitHub
  2. 2024Offline Training of Language Model Agents with Functions as Learnable Weights
  3. 2026WebFactory: Automated Compression of Foundational Language Intelligence into Grounded Web Agents
  4. 2025AgentFrontier: Expanding the Capability Frontier of LLM Agents with ZPD-Guided Data Synthesis
  5. 2024Agent-FLAN: Designing Data and Methods of Effective Agent Tuning for Large Language Models
  6. 2024Husky: A Unified, Open-Source Language Agent for Multi-Step Reasoning
  7. 2026OpenThoughts-Agent: Data Recipes for Agentic Models
  8. 2025Agent0: Unleashing Self-Evolving Agents from Zero Data via Tool-Integrated Reasoning