# AutoGuide: Automated Generation and Selection of Context-Aware Guidelines for Large Language Model Agents

arXiv: https://arxiv.org/abs/2403.08978 · HTML: https://mlsanity.com/paper/2403.08978/

## Key facts

- Cited by 74 papers in the map (data through 14 Oct 2026).
- Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use.
- Builds on 29 of its 37 references that are in the map.
- First posted on arXiv 13 Mar 2024.

## Placed in

- LLM agents and tool use / [General LLM agent frameworks and benchmarks](https://mlsanity.com/agents/general-llm-agent-frameworks-and-benchmarks/)

## Builds on (most cited first, 29 of 29)

- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [Reflexion: Language Agents with Verbal Reinforcement Learning](https://mlsanity.com/paper/2303.11366/) (2023)
- [Toolformer: Language Models Can Teach Themselves to Use Tools](https://mlsanity.com/paper/2302.04761/) (2023)
- [Self-Refine: Iterative Refinement with Self-Feedback](https://mlsanity.com/paper/2303.17651/) (2023)
- [Do As I Can, Not As I Say: Grounding Language in Robotic Affordances](https://mlsanity.com/paper/2204.01691/) (2022)
- [A Survey on Large Language Model based Autonomous Agents](https://mlsanity.com/paper/2308.11432/) (2023)
- [WebArena: A Realistic Web Environment for Building Autonomous Agents](https://mlsanity.com/paper/2307.13854/) (2023)
- [ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs](https://mlsanity.com/paper/2307.16789/) (2023)
- [The Rise and Potential of Large Language Model Based Agents: A Survey](https://mlsanity.com/paper/2309.07864/) (2023)
- [Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?](https://mlsanity.com/paper/2202.12837/) (2022)
- [Gorilla: Large Language Model Connected with Massive APIs](https://mlsanity.com/paper/2305.15334/) (2023)
- [WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agents](https://mlsanity.com/paper/2207.01206/) (2022)
- [Mind2Web: Towards a Generalist Agent for the Web](https://mlsanity.com/paper/2306.06070/) (2023)
- [ALFWorld: Aligning Text and Embodied Environments for Interactive Learning](https://mlsanity.com/paper/2010.03768/) (2020)
- [Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity](https://mlsanity.com/paper/2104.08786/) (2021)
- [Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agents](https://mlsanity.com/paper/2201.07207/) (2022)
- [ExpeL: LLM Agents Are Experiential Learners](https://mlsanity.com/paper/2308.10144/) (2023)
- [ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Tasks](https://mlsanity.com/paper/1912.01734/) (2019)
- [PAL: Program-aided Language Models](https://mlsanity.com/paper/2211.10435/) (2022)
- [GPT-4V(ision) is a Generalist Web Agent, if Grounded](https://mlsanity.com/paper/2401.01614/) (2024)
- [Language Models can Solve Computer Tasks](https://mlsanity.com/paper/2303.17491/) (2023)
- [AgentTuning: Enabling Generalized Agent Abilities for LLMs](https://mlsanity.com/paper/2310.12823/) (2023)
- [Set-of-Mark Prompting Unleashes Extraordinary Visual Grounding in GPT-4V](https://mlsanity.com/paper/2310.11441/) (2023)
- [TALM: Tool Augmented Language Models](https://mlsanity.com/paper/2205.12255/) (2022)
- [AdaPlanner: Adaptive Planning from Feedback with Language Models](https://mlsanity.com/paper/2305.16653/) (2023)
- [Grounding Language to Entities and Dynamics for Generalization in Reinforcement Learning](https://mlsanity.com/paper/2101.07393/) (2021)
- [Learning to Win by Reading Manuals in a Monte-Carlo Framework](https://mlsanity.com/paper/1401.5390/) (2014)
- [Few-shot Subgoal Planning with Language Models](https://mlsanity.com/paper/2205.14288/) (2022)

## Built on by (most cited first, 30 of 74)

- [ReasoningBank: Scaling Agent Self-Evolving with Reasoning Memory](https://mlsanity.com/paper/2509.25140/) (2025)
- [WebAgent-R1: Training Web Agents via End-to-End Multi-Turn Reinforcement Learning](https://mlsanity.com/paper/2505.16421/) (2025)
- [Evaluation and Benchmarking of LLM Agents: A Survey](https://mlsanity.com/paper/2507.21504/) (2025)
- [Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systems](https://mlsanity.com/paper/2504.01990/) (2025)
- [Beyond Browsing: API-Based Web Agents](https://mlsanity.com/paper/2410.16464/) (2024)
- [SWE-Exp: Experience-Driven Software Issue Resolution](https://mlsanity.com/paper/2507.23361/) (2025)
- [Contextual Experience Replay for Self-Improvement of Language Agents](https://mlsanity.com/paper/2506.06698/) (2025)
- [ScribeAgent: Towards Specialized Web Agents Using Production-Scale Workflow Data](https://mlsanity.com/paper/2411.15004/) (2024)
- [MPO: Boosting LLM Agents with Meta Plan Optimization](https://mlsanity.com/paper/2503.02682/) (2025)
- [From Language to Action: A Review of Large Language Models as Autonomous Agents and Tool Users](https://mlsanity.com/paper/2508.17281/) (2025)
- [THREAD: Thinking Deeper with Recursive Spawning](https://mlsanity.com/paper/2405.17402/) (2024)
- [Self-Generated In-Context Examples Improve LLM Agents for Sequential Decision-Making Tasks](https://mlsanity.com/paper/2505.00234/) (2025)
- [A Comprehensive Survey on Agent Skills: Taxonomy, Techniques, and Applications](https://mlsanity.com/paper/2605.07358/) (2026)
- [SkillDAG: Self-Evolving Typed Skill Graphs for LLM Skill Selection at Scale](https://mlsanity.com/paper/2606.03056/) (2026)
- [Enter the Mind Palace: Reasoning and Planning for Long-term Active Embodied Question Answering](https://mlsanity.com/paper/2507.12846/) (2025)
- [Formalizing Learning from Language Feedback with Provable Guarantees](https://mlsanity.com/paper/2506.10341/) (2025)
- [Experiential Reflective Learning for Self-Improving LLM Agents](https://mlsanity.com/paper/2603.24639/) (2026)
- [OpenApps: Simulating Environment Variations to Measure UI-Agent Reliability](https://mlsanity.com/paper/2511.20766/) (2025)
- [Dynamic Skill Lifecycle Management for Agentic Reinforcement Learning](https://mlsanity.com/paper/2605.10923/) (2026)
- [Recursive Experiential-Working Memory Evolution for Long-Horizon Agent Harnesses](https://mlsanity.com/paper/2608.24876/) (2026)
- [H2R: Hierarchical Hindsight Reflection for Multi-Task LLM Agents](https://mlsanity.com/paper/2509.12810/) (2025)
- [WebArbiter: A Principle-Guided Reasoning Process Reward Model for Web Agents](https://mlsanity.com/paper/2601.21872/) (2026)
- [PG-Agent: An Agent Powered by Page Graph](https://mlsanity.com/paper/2509.03536/) (2025)
- [Can LLM Agents Respond to Disasters? Benchmarking Heterogeneous Geospatial Reasoning in Emergency Operations](https://mlsanity.com/paper/2605.11633/) (2026)
- [Reflection-Based Memory For Web navigation Agents](https://mlsanity.com/paper/2506.02158/) (2025)
- [Multi-level Value Alignment in Agentic AI Systems: Survey and Perspectives](https://mlsanity.com/paper/2506.09656/) (2025)
- [Fine-tuning with RAG for Improving LLM Learning of New Skills](https://mlsanity.com/paper/2510.01375/) (2025)
- [JEF-Hinter: Leveraging Offline Knowledge for Improving Web Agents Adaptation](https://mlsanity.com/paper/2510.04373/) (2025)
- [Large Language Model Agents Are Not Always Faithful Self-Evolvers](https://mlsanity.com/paper/2601.22436/) (2026)
- [SkillRise: Agentic Reinforcement Learning for Cross-Task Skill Evolution](https://mlsanity.com/paper/2607.26784/) (2026)

## Abstract (arXiv)

Recent advances in large language models (LLMs) have empowered AI agents capable of performing various sequential decision-making tasks. However, effectively guiding LLMs to perform well in unfamiliar domains like web navigation, where they lack sufficient knowledge, has proven to be difficult with the demonstration-based in-context learning paradigm. In this paper, we introduce a novel framework, called AutoGuide, which addresses this limitation by automatically generating context-aware guidelines from offline experiences. Importantly, each context-aware guideline is expressed in concise natural language and follows a conditional structure, clearly describing the context where it is applicable. As a result, our guidelines facilitate the provision of relevant knowledge for the agent's current decision-making process, overcoming the limitations of the conventional demonstration-based learning paradigm. Our evaluation demonstrates that AutoGuide significantly outperforms competitive baselines in complex benchmark domains, including real-world web navigation.