mlsanityData to 14 Oct 2026

Autonomous Evaluation and Refinement of Digital Agents

arXiv 2404.06474 ↗First posted 9 Apr 2024cs.AIcited by 137

Not on the map. Why?

Cited by 106 in LLM agents and tool use, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We show that domain-general automatic evaluators can significantly improve the performance of agents for web navigation and device control. We experiment with multiple evaluation models that trade off between inference cost, modularity of design, and accuracy. We validate the performance of these models in several popular benchmarks for digital agents, finding between 74.4 and 92.9% agreement with oracle evaluation metrics. Finally, we use these evaluators to improve the performance of existing agents via fine-tuning and inference-time guidance. Without any additional supervision, we improve state-of-the-art performance by 29% on the popular benchmark WebArena, and achieve around 75% relative improvement in device control settings.

Key facts. Cited by 137 papers in the map (data through 14 Oct 2026). Builds on 33 of its 64 references that are in the map. First posted on arXiv 9 Apr 2024.

Builds on

33 of 64 references.

References · 33

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  3. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  4. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
  5. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,211
  6. 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 2,006
  7. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,546
  8. 2022WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agentscited by 1,251
  9. 2023Mind2Web: Towards a Generalist Agent for the Webcited by 1,242
  10. 2023CogAgent: A Visual Language Model for GUI Agentscited by 704
  11. 2024WebVoyager: Building an End-to-End Web Agent with Large Multimodal Modelscited by 438
  12. 2023AppAgent: Multimodal Agents as Smartphone Userscited by 415
21 more
  1. 2023A Real-World WebAgent with Planning, Long Context Understanding, and Program Synthesiscited by 353
  2. 2023Android in the Wild: A Large-Scale Dataset for Android Device Controlcited by 319
  3. 2024Mobile-Agent: Autonomous Multi-Modal Mobile Device Agent with Visual Perceptioncited by 293
  4. 2018Reinforcement Learning on Web Interfaces Using Workflow-Guided Explorationcited by 284
  5. 2024OS-Copilot: Towards Generalist Computer Agents with Self-Improvementcited by 205
  6. 2021RvS: What is Essential for Offline RL via Supervised Learning?cited by 185
  7. 2023You Only Look at Screens: Multimodal Chain-of-Action Agentscited by 185
  8. 2020Mapping Natural Language Instructions to Mobile UI Action Sequencescited by 181
  9. 2024UFO: A UI-Focused Agent for Windows OS Interactioncited by 148
  10. 2023Vision-Language Models as Success Detectorscited by 142
  11. 2023GPT-4V in Wonderland: Large Multimodal Models for Zero-Shot Smartphone GUI Navigationcited by 137
  12. 2022From Images to Textual Prompts: Zero-shot VQA with Frozen Large Language Modelscited by 115
  13. 2019BAIL: Best-Action Imitation Learning for Batch Deep Reinforcement Learningcited by 91
  14. 2021AndroidEnv: A Reinforcement Learning Platform for Androidcited by 90
  15. 2022A data-driven approach for learning to control computerscited by 88
  16. 2023LMRL Gym: Benchmarks for Multi-Turn Reinforcement Learning with Language Modelscited by 88
  17. 2022Enabling Conversational Interaction with Mobile UI using Large Language Modelscited by 66
  18. 2022A Dataset for Interactive Vision-Language Navigation with Unknown Command Feasibilitycited by 61
  19. 2023IdealGPT: Iteratively Decomposing Vision and Language Reasoning via Large Language Modelscited by 47
  20. 2021Grounding Open-Domain Instructions to Automate Web Support Taskscited by 43
  21. 2023Filling the Image Information Gap for VQA: Prompting Large Language Models to Proactively Ask Questionscited by 14

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 37; 2025: 60; 2026: 40.

137 papers.

Sort:

By area:

  1. 2024OpenHands: An Open Platform for AI Software Developers as Generalist Agentscited by 1,002
  2. 2024AndroidWorld: A Dynamic Benchmarking Environment for Autonomous Agentscited by 468
  3. 2024Agent Workflow Memorycited by 383
  4. 2024Training Software Engineering Agents and Verifiers with SWE-Gymcited by 336
  5. 2024Optimizing Instructions and Demonstrations for Multi-Stage Language Model Programscited by 291
  6. 2025ReasoningBank: Scaling Agent Self-Evolving with Reasoning Memorycited by 284
  7. 2024WebRL: Training LLM Web Agents via Self-Evolving Online Curriculum Reinforcement Learningcited by 256
  8. 2024Agent-as-a-Judge: Evaluate Agents with Agentscited by 238
  9. 2024Spider 2.0: Evaluating Language Models on Real-World Enterprise Text-to-SQL Workflowscited by 232
  10. 2024DigiRL: Training In-The-Wild Device-Control Agents with Autonomous Reinforcement Learningcited by 214
  11. 2024MetaMorph: Multimodal Understanding and Generation via Instruction Tuningcited by 206
  12. 2025Plan-and-Act: Improving Planning of Agents for Long-Horizon Taskscited by 195
38 more
  1. 2025SkillWeaver: Web Agents can Self-Improve by Discovering and Honing Skillscited by 192
  2. 2025An Illusion of Progress? Assessing the Current State of Web Agentscited by 174
  3. 2025WebAgent-R1: Training Web Agents via End-to-End Multi-Turn Reinforcement Learningcited by 165
  4. 2024OS-Genesis: Automating GUI Agent Trajectory Construction via Reverse Task Synthesiscited by 163
  5. 2024Fine-Tuning Large Vision-Language Models as Decision-Making Agents via Reinforcement Learningcited by 158
  6. 2024Is Your LLM Secretly a World Model of the Internet? Model-Based Planning for Web Agentscited by 154
  7. 2024Tree Search for Language Model Agentscited by 151
  8. 2024AgentTrek: Agent Trajectory Synthesis via Guiding Replay with Web Tutorialscited by 130
  9. 2024Web Agents with World Models: Learning and Leveraging Environment Dynamics in Web Navigationcited by 126
  10. 2024Dissecting Adversarial Robustness of Multimodal LM Agentscited by 120
  11. 2024AgentOccam: A Simple Yet Strong Baseline for LLM-Based Web Agentscited by 114
  12. 2025OS-Harm: A Benchmark for Measuring Safety of Computer Use Agentscited by 111
  13. 2024WebPilot: A Versatile and Autonomous Multi-Agent System for Web Task Execution with Strategic Explorationcited by 101
  14. 2025How Memory Management Impacts LLM Agents: An Empirical Study of Experience-Following Behaviorcited by 101
  15. 2025Lifelong Learning of Large Language Model based Agents: A Roadmapcited by 83
  16. 2024SPA-Bench: A Comprehensive Benchmark for SmartPhone Agent Evaluationcited by 81
  17. 2024Beyond Browsing: API-Based Web Agentscited by 78
  18. 2025Explorer: Scaling Exploration-driven Web Trajectory Synthesis for Multimodal Web Agentscited by 72
  19. 2025SEAgent: Self-Evolving Computer Use Agent with Autonomous Learning from Experiencecited by 69
  20. 2024DistRL: An Asynchronous Distributed Reinforcement Learning Framework for On-Device Control Agentscited by 68
  21. 2024Proposer-Agent-Evaluator(PAE): Autonomous Skill Discovery For Foundation Model Internet Agentscited by 50
  22. 2025Contextual Experience Replay for Self-Improvement of Language Agentscited by 48
  23. 2024Synatra: Turning Indirect Knowledge into Direct Demonstrations for Digital Agents at Scalecited by 43
  24. 2024Evaluation Agent: Efficient and Promptable Evaluation Framework for Visual Generative Modelscited by 41
  25. 2025Web-Shepherd: Advancing PRMs for Reinforcing Web Agentscited by 41
  26. 2024ScribeAgent: Towards Specialized Web Agents Using Production-Scale Workflow Datacited by 39
  27. 2025CowPilot: A Framework for Autonomous and Human-Agent Collaborative Web Navigationcited by 36
  28. 2025A Comprehensive Survey of Agents for Computer Use: Foundations, Challenges, and Future Directionscited by 34
  29. 2025Thinking vs. Doing: Agents that Reason by Scaling Test-Time Interactioncited by 34
  30. 2025Go-Browse: Training Web Agents with Structured Explorationcited by 33
  31. 2025LLM-Powered GUI Agents in Phone Automation: Surveying Progress and Prospectscited by 32
  32. 2025Look Before You Leap: A GUI-Critic-R1 Model for Pre-Operative Error Diagnosis in GUI Automationcited by 27
  33. 2026Gym-Anything: Turn any Software into an Agent Environmentcited by 27
  34. 2025Symbiotic Cooperation for Web Agents: Harnessing Complementary Strengths of Large and Small LLMscited by 24
  35. 2024Can We Rely on LLM Agents to Draft Long-Horizon Plans? Let's Take TravelPlanner as an Examplecited by 22
  36. 2025Adaptation of Agentic AI: A Survey of Post-Training, Memory, and Skillscited by 22
  37. 2026Inference-Time Scaling of Verification: Self-Evolving Deep Research Agents via Test-Time Rubric-Guided Verificationcited by 22
  38. 2025RealWebAssist: A Benchmark for Long-Horizon Web Assistance with Real-World Userscited by 21

50 most cited of 137 All 137 (JSON)