mlsanityData to 14 Oct 2026

BAGEL: Bootstrapping Agents by Guiding Exploration with Language

arXiv 2403.08140 ↗First posted 12 Mar 2024cs.CLcited by 34

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

also in 1 more
Abstract (arXiv)

Following natural language instructions by executing actions in digital environments (e.g. web-browsers and REST APIs) is a challenging task for language model (LM) agents. Unfortunately, LM agents often fail to generalize to new environments without human demonstrations. This work presents BAGEL, a method for bootstrapping LM agents without human supervision. BAGEL converts a seed set of randomly explored trajectories or synthetic instructions, into demonstrations, via round-trips between two noisy LM components: an LM labeler which converts a trajectory into a synthetic instruction, and a zero-shot LM agent which maps the synthetic instruction into a refined trajectory. By performing these round-trips iteratively, BAGEL quickly converts the initial distribution of trajectories towards those that are well-described by natural language. We use BAGEL demonstrations to adapt a zero shot LM agent at test time via in-context learning over retrieved demonstrations, and find improvements of over 2-13% absolute on ToolQA and MiniWob++, with up to 13x reduction in execution failures.

Key facts. Cited by 34 papers in the map (data through 14 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 35 of its 42 references that are in the map. First posted on arXiv 12 Mar 2024.

Builds on

35 of 42 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023AdaPlanner: Adaptive Planning from Feedback with Language Models26 May

Other references · 34

  1. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  2. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  3. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,148
  4. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,955
  5. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,348
  6. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,745
  7. 2018Know What You Don't Know: Unanswerable Questions for SQuADcited by 1,554
  8. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,434
  9. 2022Code as Policies: Language Model Programs for Embodied Controlcited by 1,210
  10. 2023PaLM 2 Technical Reportcited by 1,162
  11. 2017Hindsight Experience Replaycited by 1,136
  12. 2022Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agentscited by 1,035
22 more
  1. 2019Latent Retrieval for Weakly Supervised Open Domain Question Answeringcited by 695
  2. 2023Reinforced Self-Training (ReST) for Language Modelingcited by 500
  3. 2023Language Models can Solve Computer Taskscited by 372
  4. 2023Beyond Human Data: Scaling Self-Training for Problem-Solving with Language Modelscited by 349
  5. 2018Reinforcement Learning on Web Interfaces Using Workflow-Guided Explorationcited by 284
  6. 2023ToolQA: A Dataset for LLM Question Answering with External Toolscited by 274
  7. 2023Reward Design with Language Modelscited by 261
  8. 2022TALM: Tool Augmented Language Modelscited by 197
  9. 2023Guiding Pretraining in Reinforcement Learning with Large Language Modelscited by 187
  10. 2024WebLINX: Real-World Website Navigation with Multi-Turn Dialoguecited by 180
  11. 2023Multimodal Web Navigation with Instruction-Finetuned Foundation Modelscited by 160
  12. 2019Language as an Abstraction for Hierarchical Deep Reinforcement Learningcited by 138
  13. 2020Keep CALM and Explore: Language Models for Action Generation in Text-based Gamescited by 108
  14. 2022A data-driven approach for learning to control computerscited by 88
  15. 2017Mapping Instructions and Visual Observations to Actions with Reinforcement Learningcited by 81
  16. 2017Gated-Attention Architectures for Task-Oriented Language Groundingcited by 72
  17. 2023From Pixels to UI Actions: Learning to Follow Instructions via Graphical User Interfacescited by 68
  18. 2023SteP: Stacked LLM Policies for Web Actionscited by 66
  19. 2022Robotic Skill Acquisition via Instruction Augmentation with Vision-Language Modelscited by 58
  20. 2022Improving Intrinsic Exploration with Language Abstractionscited by 49
  21. 2023Distilling Internet-Scale Vision-Language Models into Embodied Agentscited by 30
  22. 2022Few-shot Subgoal Planning with Language Modelscited by 22

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 6; 2025: 18; 2026: 10.

34 papers.

Sort:

By area:

  1. 2024AndroidWorld: A Dynamic Benchmarking Environment for Autonomous Agentscited by 468
  2. 2024Agent Workflow Memorycited by 383
  3. 2025SkillWeaver: Web Agents can Self-Improve by Discovering and Honing Skillscited by 192
  4. 2024OS-Genesis: Automating GUI Agent Trajectory Construction via Reverse Task Synthesiscited by 163
  5. 2025Inducing Programmatic Skills for Agentic Taskscited by 111
  6. 2025AgentRewardBench: Evaluating Automatic Evaluations of Web Agent Trajectoriescited by 108
  7. 2025Agent Learning via Early Experiencecited by 85
  8. 2025WebEvolver: Enhancing Web Agent Self-Improvement with Coevolving World Modelcited by 61
  9. 2024NNetNav: Unsupervised Learning of Browser Agents Through Environment Interaction in the Wildcited by 59
  10. 2025Self-Challenging Language Model Agentscited by 59
  11. 2024Synatra: Turning Indirect Knowledge into Direct Demonstrations for Digital Agents at Scalecited by 43
  12. 2024ScribeAgent: Towards Specialized Web Agents Using Production-Scale Workflow Datacited by 39
22 more
  1. 2025A Comprehensive Survey of Agents for Computer Use: Foundations, Challenges, and Future Directionscited by 34
  2. 2025Thinking vs. Doing: Agents that Reason by Scaling Test-Time Interactioncited by 34
  3. 2025Go-Browse: Training Web Agents with Structured Explorationcited by 33
  4. 2025Accumulating Context Changes the Beliefs of Language Modelscited by 28
  5. 2026Gym-Anything: Turn any Software into an Agent Environmentcited by 27
  6. 2025Morae: Proactively Pausing UI Agents for User Choicescited by 19
  7. 2025Build the web for agents, not agents for the webcited by 18
  8. 2025Sample-Efficient Online Learning in LM Agents via Hindsight Trajectory Rewritingcited by 13
  9. 2025Automated Skill Discovery for Language Agents through Exploration and Iterative Feedbackcited by 12
  10. 2025CuES: A Curiosity-driven and Environment-grounded Synthesis Framework for Agentic RLcited by 10
  11. 2025Watch and Learn: Learning to Use Computers from Online Videoscited by 9
  12. 2026HATS: Hardness-Aware Trajectory Synthesis for GUI Agentscited by 9
  13. 2026Agentic Test-Time Scaling for WebAgentscited by 8
  14. 2025Improving Large Language Model Planning with Action Sequence Similaritycited by 7
  15. 2026Spinning Straw into Gold: Relabeling LLM Agent Trajectories in Hindsight for Successful Demonstrationscited by 5
  16. 2025Code-enabled language models can outperform reasoning models on diverse taskscited by 4
  17. 2026Structured Distillation of Web Agent Capabilities Enables Generalizationcited by 3
  18. 2026EnterpriseLab: A Full-Stack Platform for developing and deploying agents in Enterprisescited by 1
  19. 2026CORE: Code-based Inverse Self-Training Framework with Graph Expansion for Virtual Agentscited by 0
  20. 2026T2I-BiasBench: A Multi-Metric Framework for Auditing Demographic and Cultural Bias in Text-to-Image Modelscited by 0
  21. 2026Efficient Reinforcement for Visual-Textual Thinking with Discrete Diffusion Modelcited by 0
  22. 2026Efficient Test-Time Adaptation through Human-AI Interactioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Leveraging In-Context Learning for Language Model Agents
  2. 2025Improving Language Agents through BREW: Bootstrapping expeRientially-learned Environmental knoWledge
  3. 2025Natural Language Actor-Critic: Scalable Off-Policy Learning in Language Space
  4. 2026AutoTrainess: Teaching Language Models to Improve Language Models Autonomously
  5. 2026Language-based Trial and Error Falls Behind in the Era of Experience
  6. 2026ACCORD: Action-Conditioned Contextual Grounding for Language Agents
  7. 2024Offline Training of Language Model Agents with Functions as Learnable Weights
  8. 2024Words as Beacons: Guiding RL Agents with High-Level Language Prompts