mlsanityData to 14 Oct 2026

SteP: Stacked LLM Policies for Web Actions

arXiv 2310.03720 ↗First posted 5 Oct 2023cs.LGcited by 66

Not on the map. Why?

Cited by 54 in LLM agents and tool use, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Performing tasks on the web presents fundamental challenges to large language models (LLMs), including combinatorially large open-world tasks and variations across web interfaces. Simply specifying a large prompt to handle all possible behaviors and states is extremely complex, and results in behavior leaks between unrelated behaviors. Decomposition to distinct policies can address this challenge, but requires carefully handing off control between policies. We propose Stacked LLM Policies for Web Actions (SteP), an approach to dynamically compose policies to solve a diverse set of web tasks. SteP defines a Markov Decision Process where the state is a stack of policies representing the control state, i.e., the chain of policy calls. Unlike traditional methods that are restricted to static hierarchies, SteP enables dynamic control that adapts to the complexity of the task. We evaluate SteP against multiple baselines and web environments including WebArena, MiniWoB++, and a CRM. On WebArena, SteP improves (14.9% to 33.5%) over SOTA that use GPT-4 policies, while on MiniWob++, SteP is competitive with prior works while using significantly less data. Our code and data are available at https://asappresearch.github.io/webagents-step.

Key facts. Cited by 66 papers in the map (data through 14 Oct 2026). Builds on 28 of its 67 references that are in the map. First posted on arXiv 5 Oct 2023.

Builds on

28 of 67 references.

References · 28

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  4. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
  5. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
  6. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
  7. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,242
  8. 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 2,006
  9. 2022WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agentscited by 1,251
  10. 2023Mind2Web: Towards a Generalist Agent for the Webcited by 1,242
  11. 2022Code as Policies: Language Model Programs for Embodied Controlcited by 1,210
  12. 2022Inner Monologue: Embodied Reasoning through Planning with Language Modelscited by 1,081
16 more
  1. 2022Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agentscited by 1,035
  2. 2022Decomposed Prompting: A Modular Approach for Solving Complex Taskscited by 558
  3. 2022LLM-Planner: Few-Shot Grounded Planning for Embodied Agents with Large Language Modelscited by 462
  4. 2018Data-Efficient Hierarchical Reinforcement Learningcited by 455
  5. 2023Language Models can Solve Computer Taskscited by 372
  6. 2017FeUdal Networks for Hierarchical Reinforcement Learningcited by 337
  7. 2018Reinforcement Learning on Web Interfaces Using Workflow-Guided Explorationcited by 284
  8. 2023GPT4Tools: Teaching Large Language Model to Use Tools via Self-instructioncited by 248
  9. 2023AdaPlanner: Adaptive Planning from Feedback with Language Modelscited by 196
  10. 2023Synapse: Trajectory-as-Exemplar Prompting with Memory for Computer Controlcited by 187
  11. 2023Multimodal Web Navigation with Instruction-Finetuned Foundation Modelscited by 160
  12. 2016A Deep Hierarchical Approach to Lifelong Learning in Minecraftcited by 90
  13. 2022A data-driven approach for learning to control computerscited by 88
  14. 2023LASER: LLM Agent with State-Space Exploration for Web Navigationcited by 66
  15. 2023Demo2Code: From Summarizing Demonstrations to Synthesizing Code via Extended Chain-of-Thoughtcited by 39
  16. 2022Environment Generation for Zero-Shot Compositional Reinforcement Learningcited by 32

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 14; 2025: 29; 2026: 23.

66 papers.

Sort:

By area:

  1. 2024AI Agents That Mattercited by 219
  2. 2025SkillWeaver: Web Agents can Self-Improve by Discovering and Honing Skillscited by 192
  3. 2025WebAgent-R1: Training Web Agents via End-to-End Multi-Turn Reinforcement Learningcited by 165
  4. 2024The BrowserGym Ecosystem for Web Agent Researchcited by 161
  5. 2024Tree Search for Language Model Agentscited by 151
  6. 2024ST-WebAgentBench: A Benchmark for Evaluating Safety and Trustworthiness in Web Agentscited by 133
  7. 2025Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systemscited by 126
  8. 2025A Survey of Context Engineering for Large Language Modelscited by 126
  9. 2024AgentOccam: A Simple Yet Strong Baseline for LLM-Based Web Agentscited by 114
  10. 2025SafeArena: Evaluating the Safety of Autonomous Web Agentscited by 102
  11. 2024WebPilot: A Versatile and Autonomous Multi-Agent System for Web Task Execution with Strategic Explorationcited by 101
  12. 2025A Survey of WebAgents: Towards Next-Generation AI Agents for Web Automation with Large Foundation Modelscited by 87
38 more
  1. 2025Lifelong Learning of Large Language Model based Agents: A Roadmapcited by 83
  2. 2024Beyond Browsing: API-Based Web Agentscited by 78
  3. 2024ExACT: Teaching AI Agents to Explore with Reflective-MCTS and Exploratory Learningcited by 62
  4. 2024AutoManual: Constructing Instruction Manuals by LLM Agents via Interactive Environmental Learningcited by 61
  5. 2024Large Language Models Can Self-Improve At Web Agent Taskscited by 55
  6. 2025Contextual Experience Replay for Self-Improvement of Language Agentscited by 48
  7. 2025REAL: Benchmarking Autonomous Agents on Deterministic Simulations of Real Websitescited by 45
  8. 2024VideoWebArena: Evaluating Long Context Multimodal Agents with Video Understanding Web Taskscited by 43
  9. 2024ScribeAgent: Towards Specialized Web Agents Using Production-Scale Workflow Datacited by 39
  10. 2024BAGEL: Bootstrapping Agents by Guiding Exploration with Languagecited by 34
  11. 2025A Comprehensive Survey of Agents for Computer Use: Foundations, Challenges, and Future Directionscited by 34
  12. 2025Thinking vs. Doing: Agents that Reason by Scaling Test-Time Interactioncited by 34
  13. 2025A Survey of Large Language Model Empowered Agents for Recommendation and Search: Towards Next-Generation Information Retrievalcited by 33
  14. 2025LLM-Powered GUI Agents in Phone Automation: Surveying Progress and Prospectscited by 32
  15. 2026PlugMem: A Task-Agnostic Plugin Memory Module for LLM Agentscited by 32
  16. 2024Better than Your Teacher: LLM Agents that learn from Privileged AI Feedbackcited by 24
  17. 2025Symbiotic Cooperation for Web Agents: Harnessing Complementary Strengths of Large and Small LLMscited by 24
  18. 2026WebXSkill: Skill Learning for Autonomous Web Agentscited by 24
  19. 2025ProgRM: Build Better GUI Agents with Progress Rewardscited by 21
  20. 2025Build the web for agents, not agents for the webcited by 18
  21. 2025FocusAgent: Simple Yet Effective Ways of Trimming the Large Context of Web Agentscited by 16
  22. 2025WAREX: Web Agent Reliability Evaluation on Existing Benchmarkscited by 15
  23. 2025R2D2: Remembering, Replaying and Dynamic Decision Making with a Reflective Agentic Memorycited by 14
  24. 2025LiteWebAgent: The Open-Source Suite for VLM-Based Web-Agent Applicationscited by 14
  25. 2026AgentHER: Hindsight Experience Replay for LLM Agent Trajectory Relabelingcited by 13
  26. 2025OpenApps: Simulating Environment Variations to Measure UI-Agent Reliabilitycited by 12
  27. 2026From Question Answering to Task Completion: A Survey on Agent System and Harness Designcited by 12
  28. 2026OpAgent: Operator Agent for Web Navigationcited by 8
  29. 2025Branch-and-Browse: Efficient and Controllable Web Exploration with Tree-Structured Reasoning and Action Memorycited by 6
  30. 2026ColorBrowserAgent: Complex Long-Horizon Browser Agent with Adaptive Knowledge Evolutioncited by 6
  31. 2025Reflection-Based Memory For Web navigation Agentscited by 5
  32. 2025Doc2Agent: Scalable Generation of Tool-Using Agents from API Documentationcited by 5
  33. 2025LineRetriever: Planning-Aware Observation Reduction for Web Agentscited by 5
  34. 2025WebDART: Dynamic Decomposition and Re-planning for Complex Web Taskscited by 5
  35. 2026Beyond State Consistency: Behavior Consistency in Text-Based World Modelscited by 5
  36. 2026The Tool Illusion: Rethinking Tool Use in Web Agentscited by 4
  37. 2025WEBSERV: A Full-Stack and RL-Ready Web Environment for Training Web Agents at Scalecited by 3
  38. 2026AI Planning Framework for LLM-Based Web Agentscited by 3

50 most cited of 66 All 66 (JSON)