mlsanityData to 13 Oct 2026

A Survey on Complex Tasks for Goal-Directed Interactive Agents

arXiv 2409.18538 ↗First posted 27 Sep 2024cs.CLcited by 2

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

Goal-directed interactive agents, which autonomously complete tasks through interactions with their environment, can assist humans in various domains of their daily lives. Recent advances in large language models (LLMs) led to a surge of new, more and more challenging tasks to evaluate such agents. To properly contextualize performance across these tasks, it is imperative to understand the different challenges they pose to agents. To this end, this survey compiles relevant tasks and environments for evaluating goal-directed interactive agents, structuring them along dimensions relevant for understanding current obstacles. An up-to-date compilation of relevant resources can be found on our project website: https://coli-saar.github.io/interactive-agents.

Key facts. Cited by 2 papers in the map (data through 13 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 75 of its 99 references that are in the map. First posted on arXiv 27 Sep 2024.

Builds on

75 of 99 references.

Closest ancestors · 5

Same direction, newest first

  1. 2024AgentBoard: An Analytical Evaluation Board of Multi-turn LLM Agents24 Jan
  2. 2024Exploring Large Language Model based Intelligent Agents: Definitions, Methods, and Prospects7 Jan
  3. 2023Large Language Models Empowered Agent-based Modeling and Simulation: A Survey and Perspectives19 Dec
  4. 2023Language Models, Agent Models, and World Models: The LAW for Machine Reasoning and Planning8 Dec
  5. 2023The Rise and Potential of Large Language Model Based Agents: A Survey14 Sep

Other references · 70

  1. 2020Language Models are Few-Shot Learnerscited by 32,320
  2. 2021Training Verifiers to Solve Math Word Problemscited by 9,916
  3. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,502
  4. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,117
  5. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,716
  6. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,274
  7. 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 1,953
  8. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,904
  9. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,531
  10. 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,313
  11. 2023Mind2Web: Towards a Generalist Agent for the Webcited by 1,222
  12. 2022WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agentscited by 1,219
38 more
  1. 2020ALFWorld: Aligning Text and Embodied Environments for Interactive Learningcited by 1,167
  2. 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environmentscited by 1,154
  3. 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memoriescited by 1,111
  4. 2023GAIA: a benchmark for General AI Assistantscited by 1,102
  5. 2024OpenHands: An Open Platform for AI Software Developers as Generalist Agentscited by 977
  6. 2017AI2-THOR: An Interactive 3D Environment for Visual AIcited by 859
  7. 2019ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Taskscited by 775
  8. 2018MultiWOZ -- A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modellingcited by 761
  9. 2017Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learningcited by 687
  10. 2024A Survey on the Memory Mechanism of Large Language Model based Agentscited by 625
  11. 2023ChemCrow: Augmenting large-language models with chemistry toolscited by 622
  12. 2023API-Bank: A Comprehensive Benchmark for Tool-Augmented LLMscited by 583
  13. 2023Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Modelscited by 557
  14. 2022Decomposed Prompting: A Modular Approach for Solving Complex Taskscited by 556
  15. 2024Executable Code Actions Elicit Better LLM Agentscited by 548
  16. 2015Compositional Semantic Parsing on Semi-Structured Tablescited by 513
  17. 2022Visual Programming: Compositional visual reasoning without trainingcited by 510
  18. 2022MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledgecited by 473
  19. 2023ToolAlpaca: Generalized Tool Learning for Language Models with 3000 Simulated Casescited by 387
  20. 2024AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agentscited by 377
  21. 2017Embodied Question Answeringcited by 335
  22. 2023Minigrid & Miniworld: Modular & Customizable Reinforcement Learning Environments for Goal-Oriented Taskscited by 322
  23. 2018TextWorld: A Learning Environment for Text-based Gamescited by 306
  24. 2023Set-of-Mark Prompting Unleashes Extraordinary Visual Grounding in GPT-4Vcited by 299
  25. 2023Large Language Models as Tool Makerscited by 294
  26. 2023MINT: Evaluating LLMs in Multi-turn Interaction with Tools and Language Feedbackcited by 294
  27. 2023ToolQA: A Dataset for LLM Question Answering with External Toolscited by 274
  28. 2020HybridQA: A Dataset of Multi-Hop Question Answering over Tabular and Textual Datacited by 264
  29. 2024LLMs instead of Human Judges? A Large Scale Empirical Study across 20 NLP Evaluation Taskscited by 239
  30. 2022Pre-Trained Language Models for Interactive Decision-Makingcited by 221
  31. 2020The NetHack Learning Environmentcited by 197
  32. 2020Mapping Natural Language Instructions to Mobile UI Action Sequencescited by 180
  33. 2021FeTaQA: Free-form Table Question Answeringcited by 175
  34. 2023ADaPT: As-Needed Decomposition and Planning with Language Modelscited by 174
  35. 2019Interactive Fiction Games: A Colossal Adventurecited by 166
  36. 2019A Survey of Reinforcement Learning Informed by Natural Languagecited by 154
  37. 2023TaskBench: Benchmarking Large Language Models for Task Automationcited by 141
  38. 2024InfiAgent-DABench: Evaluating Agents on Data Analysis Taskscited by 135

50 most cited of 70 All 70 (JSON)

Built on by

Cited by year

Citing papers per year: 2025: 2.

2 papers.

  1. 2025A Survey of Context Engineering for Large Language Modelscited by 122
  2. 2025Collaborative Problem-Solving in an Optimization Gamecited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2026A Behavioural and Representational Evaluation of Goal-Directedness in Language Model Agents
  2. 2024AgentSense: Benchmarking Social Intelligence of Language Agents through Interactive Scenarios
  3. 2024Multi-Agent Large Language Models for Conversational Task-Solving
  4. 2025Enhancing LLM-Based Agents via Global Planning and Hierarchical Execution
  5. 2024Goal Inference from Open-Ended Dialog
  6. 2025The Behavior Gap: Evaluating Zero-shot LLM Agents in Complex Task-Oriented Dialogs
  7. 2023Describe, Explain, Plan and Select: Interactive Planning with Large Language Models Enables Open-World Multi-Task Agents
  8. 2024Sub-goal Distillation: A Method to Improve Small Language Agents