# Pangu-Agent: A Fine-Tunable Generalist Agent with Structured Reasoning

arXiv: https://arxiv.org/abs/2312.14878 · HTML: https://mlsanity.com/paper/2312.14878/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning.
- Builds on 25 of its 96 references that are in the map.
- First posted on arXiv 22 Dec 2023.

## Placed in

- LLM reasoning / [Reasoning for agents, planning and code](https://mlsanity.com/lr/reasoning-for-agents-planning-and-code/)
- LLM agents and tool use / [General LLM agent frameworks and benchmarks](https://mlsanity.com/agents/general-llm-agent-frameworks-and-benchmarks/)

## Builds on (most cited first, 25 of 25)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [Reflexion: Language Agents with Verbal Reinforcement Learning](https://mlsanity.com/paper/2303.11366/) (2023)
- [Self-Refine: Iterative Refinement with Self-Feedback](https://mlsanity.com/paper/2303.17651/) (2023)
- [Tree of Thoughts: Deliberate Problem Solving with Large Language Models](https://mlsanity.com/paper/2305.10601/) (2023)
- [Mistral 7B](https://mlsanity.com/paper/2310.06825/) (2023)
- [HuggingFace's Transformers: State-of-the-art Natural Language Processing](https://mlsanity.com/paper/1910.03771/) (2019)
- [AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation](https://mlsanity.com/paper/2308.08155/) (2023)
- [ALFWorld: Aligning Text and Embodied Environments for Interactive Learning](https://mlsanity.com/paper/2010.03768/) (2020)
- [Large Language Models Cannot Self-Correct Reasoning Yet](https://mlsanity.com/paper/2310.01798/) (2023)
- [Graph of Thoughts: Solving Elaborate Problems with Large Language Models](https://mlsanity.com/paper/2308.09687/) (2023)
- [Reasoning with Language Model is Planning with World Model](https://mlsanity.com/paper/2305.14992/) (2023)
- [A Survey on In-context Learning](https://mlsanity.com/paper/2301.00234/) (2022)
- [OpenChat: Advancing Open-source Language Models with Mixed-Quality Data](https://mlsanity.com/paper/2309.11235/) (2023)
- [Large Language Model Guided Tree-of-Thought](https://mlsanity.com/paper/2305.08291/) (2023)
- [Playing repeated games with Large Language Models](https://mlsanity.com/paper/2305.16867/) (2023)
- [ReWOO: Decoupling Reasoning from Observations for Efficient Augmented Language Models](https://mlsanity.com/paper/2305.18323/) (2023)
- [Chain of Hindsight Aligns Language Models with Feedback](https://mlsanity.com/paper/2302.02676/) (2023)
- [GPT-4 Doesn't Know It's Wrong: An Analysis of Iterative Prompting for Reasoning Problems](https://mlsanity.com/paper/2310.12397/) (2023)
- [BOLAA: Benchmarking and Orchestrating LLM-augmented Autonomous Agents](https://mlsanity.com/paper/2308.05960/) (2023)
- [Limitations of Autoregressive Models and Their Alternatives](https://mlsanity.com/paper/2010.11939/) (2020)
- [Beyond Chain-of-Thought, Effective Graph-of-Thought Reasoning in Language Models](https://mlsanity.com/paper/2305.16582/) (2023)
- [Learning High-level Representations from Demonstrations](https://mlsanity.com/paper/1802.06604/) (2018)

## Built on by (most cited first, 30 of 30)

- [A Multimodal Foundation Agent for Financial Trading: Tool-Augmented, Diversified, and Generalist](https://mlsanity.com/paper/2402.18485/) (2024)
- [Memento: Fine-tuning LLM Agents without Fine-tuning LLMs](https://mlsanity.com/paper/2508.16153/) (2025)
- [DS-Agent: Automated Data Science by Empowering Large Language Models with Case-Based Reasoning](https://mlsanity.com/paper/2402.17453/) (2024)
- [AgentGym: Evolving Large Language Model-based Agents across Diverse Environments](https://mlsanity.com/paper/2406.04151/) (2024)
- [Direct Multi-Turn Preference Optimization for Language Agents](https://mlsanity.com/paper/2406.14868/) (2024)
- [Agents in Software Engineering: Survey, Landscape, and Vision](https://mlsanity.com/paper/2409.09030/) (2024)
- [DynaSaur: Large Language Agents Beyond Predefined Actions](https://mlsanity.com/paper/2411.01747/) (2024)
- [Enhancing Decision-Making for LLM Agents via Step-Level Q-Value Models](https://mlsanity.com/paper/2409.09345/) (2024)
- [CMAT: A Multi-Agent Collaboration Tuning Framework for Enhancing Small Language Models](https://mlsanity.com/paper/2404.01663/) (2024)
- [Aviary: training language agents on challenging scientific tasks](https://mlsanity.com/paper/2412.21154/) (2024)
- [Kolb-Based Experiential Learning for Generalist Agents with Human-Level Kaggle Data Science Performance](https://mlsanity.com/paper/2411.03562/) (2024)
- [Can We Rely on LLM Agents to Draft Long-Horizon Plans? Let's Take TravelPlanner as an Example](https://mlsanity.com/paper/2408.06318/) (2024)
- [Agentic Information Retrieval](https://mlsanity.com/paper/2410.09713/) (2024)
- [Natural Language Reinforcement Learning](https://mlsanity.com/paper/2402.07157/) (2024)
- [Reinforcing Language Agents via Policy Optimization with Action Decomposition](https://mlsanity.com/paper/2405.15821/) (2024)
- [Tree-OPO: Off-policy Monte Carlo Tree-Guided Advantage Optimization for Multistep Reasoning](https://mlsanity.com/paper/2509.09284/) (2025)
- [Cognitive Kernel: An Open-source Agent System towards Generalist Autopilots](https://mlsanity.com/paper/2409.10277/) (2024)
- [A call for embodied AI](https://mlsanity.com/paper/2402.03824/) (2024)
- [TimeWarp: Evaluating Web Agents by Revisiting the Past](https://mlsanity.com/paper/2603.04949/) (2026)
- [Memento 2: Learning by Stateful Reflective Memory](https://mlsanity.com/paper/2512.22716/) (2025)
- [Privacy Collapse: Benign Fine-Tuning Can Break Contextual Privacy in Language Models](https://mlsanity.com/paper/2601.15220/) (2026)
- [Natural Language Reinforcement Learning](https://mlsanity.com/paper/2411.14251/) (2024)
- [Uncertainty in Action: Confidence Elicitation in Embodied Agents](https://mlsanity.com/paper/2503.10628/) (2025)
- [Flow-of-Options: Diversified and Improved LLM Reasoning by Thinking Through Options](https://mlsanity.com/paper/2502.12929/) (2025)
- [DeepSeek: Paradigm Shifts and Technical Evolution in Large AI Models](https://mlsanity.com/paper/2507.09955/) (2025)
- [Rethinking Large Language Model Distillation: A Constrained Markov Decision Process Perspective](https://mlsanity.com/paper/2509.22921/) (2025)
- [ST-Lite: Training-Free KV Cache Compression with Spatio-Trajectory Guidance for Long-Horizon GUI Agents](https://mlsanity.com/paper/2603.00188/) (2026)
- [Reasoning Systems as Structured Processes: Foundations, Failures, and Formal Criteria](https://mlsanity.com/paper/2508.01763/) (2025)
- [Succeed or Learn Slowly: Sample Efficient Off-Policy Reinforcement Learning for Mobile App Control](https://mlsanity.com/paper/2509.01720/) (2025)
- [Does This Action Still Explain the Task? Reverse Scoring for Diffusion Language Model Agents](https://mlsanity.com/paper/2609.38536/) (2026)

## Abstract (arXiv)

A key method for creating Artificial Intelligence (AI) agents is Reinforcement Learning (RL). However, constructing a standalone RL policy that maps perception to action directly encounters severe problems, chief among them being its lack of generality across multiple tasks and the need for a large amount of training data. The leading cause is that it cannot effectively integrate prior information into the perception-action cycle when devising the policy. Large language models (LLMs) emerged as a fundamental way to incorporate cross-domain knowledge into AI agents but lack crucial learning and adaptation toward specific decision problems. This paper presents a general framework model for integrating and learning structured reasoning into AI agents' policies. Our methodology is motivated by the modularity found in the human brain. The framework utilises the construction of intrinsic and extrinsic functions to add previous understandings of reasoning structures. It also provides the adaptive ability to learn models inside every module or function, consistent with the modular structure of cognitive processes. We describe the framework in-depth and compare it with other AI pipelines and existing frameworks. The paper explores practical applications, covering experiments that show the effectiveness of our method. Our results indicate that AI agents perform and adapt far better when organised reasoning and prior knowledge are embedded. This opens the door to more resilient and general AI agent systems.