mlsanityData to 14 Oct 2026

Auto-GPT for Online Decision Making: Benchmarks and Additional Opinions

arXiv 2306.02224 ↗First posted 4 Jun 2023cs.AI cs.LGcited by 255

Not on the map. Why?

Cited by 158 in LLM agents and tool use, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Auto-GPT is an autonomous agent that leverages recent advancements in adapting Large Language Models (LLMs) for decision-making tasks. While there has been a growing interest in Auto-GPT stypled agents, questions remain regarding the effectiveness and flexibility of Auto-GPT in solving real-world decision-making tasks. Its limited capability for real-world engagement and the absence of benchmarks contribute to these uncertainties. In this paper, we present a comprehensive benchmark study of Auto-GPT styled agents in decision-making tasks that simulate real-world scenarios. Our aim is to gain deeper insights into this problem and understand the adaptability of GPT-based agents. We compare the performance of popular LLMs such as GPT-4, GPT-3.5, Claude, and Vicuna in Auto-GPT styled decision-making tasks. Furthermore, we introduce the Additional Opinions algorithm, an easy and effective method that incorporates supervised/imitation-based learners into the Auto-GPT scheme. This approach enables lightweight supervised learning without requiring fine-tuning of the foundational LLMs. We demonstrate through careful baseline comparisons and ablation studies that the Additional Opinions algorithm significantly enhances performance in online decision-making benchmarks, including WebShop and ALFWorld.

Key facts. Cited by 255 papers in the map (data through 14 Oct 2026). Builds on 19 of its 30 references that are in the map. First posted on arXiv 4 Jun 2023.

Builds on

19 of 30 references.

References · 19

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  4. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  5. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  6. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  7. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,568
  8. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
  9. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,790
  10. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,592
  11. 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,332
  12. 2022WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agentscited by 1,251
7 more
  1. 2020ALFWorld: Aligning Text and Embodied Environments for Interactive Learningcited by 1,205
  2. 2023HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Facecited by 1,190
  3. 2019ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Taskscited by 783
  4. 2022Complexity-Based Prompting for Multi-Step Reasoningcited by 388
  5. 2018TextWorld: A Learning Environment for Text-based Gamescited by 310
  6. 2023TaskMatrix.AI: Completing Tasks by Connecting Foundation Models with Millions of APIscited by 184
  7. 2023Foundation Models for Decision Making: Problems, Methods, and Opportunitiescited by 164

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 23; 2024: 69; 2025: 107; 2026: 56.

255 papers.

Sort:

By area:

  1. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,212
  2. 2023GAIA: a benchmark for General AI Assistantscited by 1,128
  3. 2025UI-TARS: Pioneering Automated GUI Interaction with Native Agentscited by 610
  4. 2024From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judgecited by 494
  5. 2023AppAgent: Multimodal Agents as Smartphone Userscited by 415
  6. 2024Mobile-Agent: Autonomous Multi-Modal Mobile Device Agent with Visual Perceptioncited by 293
  7. 2023DriveMLM: Aligning Multi-Modal Large Language Models with Behavioral Planning States for Autonomous Drivingcited by 233
  8. 2024Magentic-One: A Generalist Multi-Agent System for Solving Complex Taskscited by 232
  9. 2024TradingAgents: Multi-Agents LLM Financial Trading Frameworkcited by 188
  10. 2023Navigate through Enigmatic Labyrinth A Survey of Chain of Thought Reasoning: Advances, Frontiers and Futurecited by 184
  11. 2025AI Agents vs. Agentic AI: A Conceptual Taxonomy, Applications and Challengescited by 169
  12. 2024Recursive Introspection: Teaching Language Model Agents How to Self-Improvecited by 165
38 more
  1. 2024The BrowserGym Ecosystem for Web Agent Researchcited by 161
  2. 2024BALROG: Benchmarking Agentic LLM and VLM Reasoning On Gamescited by 146
  3. 2024A Multimodal Foundation Agent for Financial Trading: Tool-Augmented, Diversified, and Generalistcited by 137
  4. 2024GenArtist: Multimodal LLM as an Agent for Unified Image Generation and Editingcited by 133
  5. 2024BadAgent: Inserting and Activating Backdoor Attacks in LLM Agentscited by 115
  6. 2025Agentic Reasoning: A Streamlined Framework for Enhancing LLM Reasoning with Agentic Toolscited by 106
  7. 2025Memp: Exploring Agent Procedural Memorycited by 103
  8. 2025Towards Scientific Intelligence: A Survey of LLM-based Scientific Agentscited by 98
  9. 2024CodeR: Issue Resolving with Multi-Agent and Task Graphscited by 91
  10. 2024I-Design: Personalized LLM Interior Designercited by 87
  11. 2024From Individual to Society: A Survey on Social Simulation Driven by Large Language Model-based Agentscited by 82
  12. 2024PsySafe: A Comprehensive Framework for Psychological-based Attack, Defense, and Evaluation of Multi-agent System Safetycited by 80
  13. 2025Agentic AI for Scientific Discovery: A Survey of Progress, Challenges, and Future Directionscited by 77
  14. 2025AgentPRM: Process Reward Models for LLM Agents via Step-Wise Promise and Progresscited by 77
  15. 2023ChessGPT: Bridging Policy Learning and Language Modelingcited by 75
  16. 2025Design Patterns for Securing LLM Agents against Prompt Injectionscited by 73
  17. 2024MLLM-Tool: A Multimodal Large Language Model For Tool Agent Learningcited by 69
  18. 2025SE-Agent: Self-Evolution Trajectory Optimization in Multi-Step Reasoning with LLM-Based Agentscited by 69
  19. 2024DistRL: An Asynchronous Distributed Reinforcement Learning Framework for On-Device Control Agentscited by 68
  20. 2023LASER: LLM Agent with State-Space Exploration for Web Navigationcited by 66
  21. 2023MechAgents: Large language model multi-agent collaborations can solve mechanics problems, generate new data, and integrate knowledgecited by 63
  22. 2024ExACT: Teaching AI Agents to Explore with Reflective-MCTS and Exploratory Learningcited by 62
  23. 2024When Search Engine Services meet Large Language Models: Visions and Challengescited by 59
  24. 2025TRiSM for Agentic AI: A Review of Trust, Risk, and Security Management in LLM-based Agentic Multi-Agent Systemscited by 59
  25. 2024Benchmarking Mobile Device Control Agents across Diverse Configurationscited by 55
  26. 2024Self-evolving Agents with reflective and memory-augmented abilitiescited by 52
  27. 2024You Name It, I Run It: An LLM Agent to Execute Tests of Arbitrary Projectscited by 47
  28. 2025AppAgentX: Evolving GUI Agents as Proficient Smartphone Userscited by 46
  29. 2023Large Language Models are Zero Shot Hypothesis Proposerscited by 45
  30. 2025Agentic Web: Weaving the Next Web with AI Agentscited by 45
  31. 2023GameGPT: Multi-agent Collaborative Framework for Game Developmentcited by 43
  32. 2025R&D-Agent-Quant: A Multi-Agent Framework for Data-Centric Factors and Model Joint Optimizationcited by 43
  33. 2025Vibe Coding vs. Agentic Coding: Fundamentals and Practical Implications of Agentic AIcited by 42
  34. 2024Adaptive In-conversation Team Building for Language Model Agentscited by 39
  35. 2023SiDA-MoE: Sparsity-Inspired Data-Aware Serving for Efficient and Scalable Large Mixture-of-Experts Modelscited by 38
  36. 2024Richelieu: Self-Evolving LLM-Based Agents for AI Diplomacycited by 36
  37. 2024Agents Thinking Fast and Slow: A Talker-Reasoner Architecturecited by 36
  38. 2024DreamFactory: Pioneering Multi-Scene Long Video Generation with a Multi-Agent Frameworkcited by 35

50 most cited of 255 All 255 (JSON)