mlsanityData to 7 Oct 2026

AMOR: A Recipe for Building Adaptable Modular Knowledge Agents Through Process Feedback

arXiv 2402.01469 ↗First posted 2 Feb 2024cs.CLcited by 25

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

The notable success of large language models (LLMs) has sparked an upsurge in building language agents to complete various complex tasks. We present AMOR, an agent framework based on open-source LLMs, which reasons with external knowledge bases and adapts to specific domains through human supervision to the reasoning process. AMOR builds reasoning logic over a finite state machine (FSM) that solves problems through autonomous executions and transitions over disentangled modules. This allows humans to provide direct feedback to the individual modules, and thus naturally forms process supervision. Based on this reasoning and feedback framework, we develop AMOR through two-stage fine-tuning: warm-up and adaptation. The former fine-tunes the LLM with examples automatically constructed from various public datasets, enabling AMOR to generalize across different knowledge environments, while the latter tailors AMOR to specific domains using process feedback. Extensive experiments across multiple domains demonstrate the advantage of AMOR to strong baselines, thanks to its FSM-based reasoning and process feedback mechanism. The code and data are publicly available at urlhttps://github.com/JianGuanTHU/AMOR.

Key facts. Cited by 25 papers in the map (data through 7 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 46 of its 57 references that are in the map. First posted on arXiv 2 Feb 2024.

Builds on

46 of 57 references.

Closest ancestors · 6

Same direction, newest first

  1. 2023Agent Lumos: Unified and Modular Training for Open-Source Language Agents9 Nov
  2. 2023AgentTuning: Enabling Generalized Agent Abilities for LLMs19 Oct
  3. 2023FireAct: Toward Language Agent Fine-tuning9 Oct
  4. 2023A Survey on Large Language Model based Autonomous Agents22 Aug
  5. 2023AgentBench: Evaluating LLMs as Agents7 Aug
  6. 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework1 Aug

Other references · 40

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  5. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  6. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  7. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  8. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  9. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  10. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  11. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  12. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
28 more
  1. 2023Let's Verify Step by Stepcited by 3,598
  2. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
  3. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
  4. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  5. 2022Emergent Abilities of Large Language Modelscited by 2,483
  6. 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,140
  7. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
  8. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,880
  9. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,800
  10. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,722
  11. 2019PubMedQA: A Dataset for Biomedical Research Question Answeringcited by 1,271
  12. 2020Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Stepscited by 1,259
  13. 2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,203
  14. 2022Measuring and Narrowing the Compositionality Gap in Language Modelscited by 1,054
  15. 2022Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questionscited by 911
  16. 2021MuSiQue: Multihop Questions via Single-hop Question Compositioncited by 893
  17. 2023Graph of Thoughts: Solving Elaborate Problems with Large Language Modelscited by 873
  18. 2022Solving math word problems with process- and outcome-based feedbackcited by 871
  19. 2023Large Language Models Can Be Easily Distracted by Irrelevant Contextcited by 820
  20. 2022Decomposed Prompting: A Modular Approach for Solving Complex Taskscited by 554
  21. 2023Active Retrieval Augmented Generationcited by 506
  22. 2019Neural Text Generation with Unlikelihood Trainingcited by 489
  23. 2021A Dataset of Information-Seeking Questions and Answers Anchored in Research Paperscited by 475
  24. 2024Contrastive Preference Optimization: Pushing the Boundaries of LLM Performance in Machine Translationcited by 390
  25. 2023A Real-World WebAgent with Planning, Long Context Understanding, and Program Synthesiscited by 350
  26. 2023ReWOO: Decoupling Reasoning from Observations for Efficient Augmented Language Modelscited by 174
  27. 2024In-context Learning with Retrieved Demonstrations for Language Models: A Surveycited by 69
  28. 2023Language Models Hallucinate, but May Excel at Fact Verificationcited by 35

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 7; 2025: 17; 2026: 1.

25 papers.

Sort:

By area:

  1. 2025Reinforcing Spatial Reasoning in Vision-Language Models with Interwoven Thinking and Visual Drawingcited by 130
  2. 2025Lifelong Learning of Large Language Model based Agents: A Roadmapcited by 78
  3. 2024Agent Planning with World Knowledge Modelcited by 66
  4. 2024KnowAgent: Knowledge-Augmented Planning for LLM-Based Agentscited by 61
  5. 2025A Survey on the Optimization of Large Language Model-based Agentscited by 45
  6. 2025A Survey on Personalized Alignment -- The Missing Piece for Large Language Models in Real-World Applicationscited by 40
  7. 2024Benchmarking Agentic Workflow Generationcited by 36
  8. 2025MPO: Boosting LLM Agents with Meta Plan Optimizationcited by 36
  9. 2024Unlocking Reasoning Potential in Large Langauge Models by Scaling Code-form Planningcited by 24
  10. 2024From the Least to the Most: Building a Plug-and-Play Visual Reasoner via Data Synthesiscited by 17
  11. 2025A Goal Without a Plan Is Just a Wish: Efficient and Effective Global Planner Training for Long-Horizon Agent Taskscited by 17
  12. 2025RAS: Retrieval-And-Structuring for Knowledge-Intensive LLM Generationcited by 16
13 more
  1. 2025a1: Steep Test-time Scaling Law via Environment Augmented Generationcited by 13
  2. 2024CheckEmbed: Effective Verification of LLM Solutions to Open-Ended Taskscited by 10
  3. 2025Agentic Knowledgeable Self-awarenesscited by 8
  4. 2025FinMMR: Make Financial Numerical Reasoning More Multimodal, Comprehensive, and Challengingcited by 6
  5. 2025Extended Inductive Reasoning for Personalized Preference Inference from Behavioral Signalscited by 5
  6. 2025EDGE: Efficient Data Selection for LLM Agents via Guideline Effectivenesscited by 4
  7. 2025Putting on the Thinking Hats: A Survey on Chain of Thought Fine-tuning from the Perspective of Human Reasoning Mechanismcited by 4
  8. 20242D-TPE: Two-Dimensional Positional Encoding Enhances Table Understanding for Large Language Modelscited by 2
  9. 2025GRITHopper: Decomposition-Free Multi-Hop Dense Retrievalcited by 2
  10. 2025Toward Real-World Table Agents: Capabilities, Workflows, and Design Principles for LLM-based Table Intelligencecited by 2
  11. 2025AutoContext: Instance-Level Context Learning for LLM Agentscited by 1
  12. 2025A Tale of LLMs and Induced Small Proxies: Scalable Small Language Models for Knowledge Miningcited by 0
  13. 2026GraphThink: Graph-Enhanced LLM Thinking for Long-Horizon Embodied Task Planningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026A Comprehensive Evaluation of LLM Reasoning: From Single-Model to Multi-Agent Paradigms
  2. 2024Sibyl: Simple yet Effective Agent Framework for Complex Real-world Reasoning
  3. 2025Adaptive Minds: Empowering Agents with LoRA-as-Tools
  4. 2024LLM2: Let Large Language Models Harness System 2 Reasoning
  5. 2024Textualized Agent-Style Reasoning for Complex Tasks by Multiple Round LLM Generation
  6. 2025A Survey of Frontiers in LLM Reasoning: Inference Scaling, Learning to Reason, and Agentic Systems
  7. 2025From Language to Action: A Review of Large Language Models as Autonomous Agents and Tool Users
  8. 2025Adaptive Multi-Agent Reasoning via Automated Workflow Generation