mlsanityData to 14 Oct 2026

Improving Grounded Language Understanding in a Collaborative Environment by Interacting with Agents Through Help Feedback

arXiv 2304.10750 ↗First posted 21 Apr 2023cs.CL cs.AIcited by 32

Not on the map. Why?

Cited by 21 in LLM agents and tool use, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Many approaches to Natural Language Processing (NLP) tasks often treat them as single-step problems, where an agent receives an instruction, executes it, and is evaluated based on the final outcome. However, human language is inherently interactive, as evidenced by the back-and-forth nature of human conversations. In light of this, we posit that human-AI collaboration should also be interactive, with humans monitoring the work of AI agents and providing feedback that the agent can understand and utilize. Further, the AI agent should be able to detect when it needs additional information and proactively ask for help. Enabling this scenario would lead to more natural, efficient, and engaging human-AI collaborations. In this work, we explore these directions using the challenging task defined by the IGLU competition, an interactive grounded language understanding task in a MineCraft-like world. We explore multiple types of help players can give to the AI to guide it and analyze the impact of this help in AI behavior, resulting in performance improvements.

Key facts. Cited by 32 papers in the map (data through 14 Oct 2026). Builds on 29 of its 59 references that are in the map. First posted on arXiv 21 Apr 2023.

Builds on

29 of 59 references.

References · 29

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  3. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
  4. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,568
  5. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
  6. 2019ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Taskscited by 783
  7. 2020A Survey of Deep Active Learningcited by 554
  8. 2022MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledgecited by 478
  9. 2023REFINER: Reasoning Feedback on Intermediate Representationscited by 205
  10. 2021TEACh: Task-driven Embodied Agents that Chatcited by 197
  11. 2021Building and Evaluating Open-Domain Dialogue Corpora with Clarifying Questionscited by 63
  12. 2020ConvAI3: Generating Clarifying Questions for Open-Domain Dialogue Systems (ClariQ)cited by 58
17 more
  1. 2020Imitating Interactive Intelligencecited by 55
  2. 2021Learning to Repair: Repairing model output errors after deployment using a dynamic memory of feedbackcited by 35
  3. 2021NL-EDIT: Correcting semantic parse errors through natural language interactioncited by 33
  4. 2022Learning to Execute Actions or Ask Clarification Questionscited by 30
  5. 2019CraftAssist: A Framework for Dialogue-enabled Interactive Agentscited by 28
  6. 2021The MineRL BASALT Competition on Learning from Human Feedbackcited by 24
  7. 2022Interactive Grounded Language Understanding in a Collaborative Environment: IGLU 2021cited by 23
  8. 2021Open-domain clarification question generation without question examplescited by 20
  9. 2021NeurIPS 2021 Competition IGLU: Interactive Grounded Language Understanding in a Collaborative Environmentcited by 15
  10. 2022IGLU 2022: Interactive Grounded Language Understanding in a Collaborative Environment at NeurIPS 2022cited by 14
  11. 2022Collecting Interactive Multi-modal Datasets for Grounded Language Understandingcited by 10
  12. 2019Improving Natural Language Interaction with Robots Using Advicecited by 9
  13. 2022IGLU Gridworld: Simple and Fast Environment for Embodied Dialog Agentscited by 9
  14. 2022Learning to Solve Voxel Building Embodied Tasks from Pixels and Natural Language Instructionscited by 9
  15. 2022Unsupervised Question Clarity Prediction Through Retrieved Item Coherencycited by 8
  16. 2021Improving Neural Model Performance through Natural Language Feedback on Their Explanationscited by 5
  17. 2021Deciding Whether to Ask Clarifying Questions in Large-Scale Spoken Language Understandingcited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 5; 2024: 10; 2025: 14; 2026: 3.

32 papers.

Sort:

By area:

  1. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,242
  2. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,439
  3. 2024Exploring Large Language Model based Intelligent Agents: Definitions, Methods, and Prospectscited by 132
  4. 2024AI Agents Under Threat: A Survey of Key Security Challenges and Future Pathwayscited by 127
  5. 2025A Survey on Code Generation with LLM-based Agentscited by 106
  6. 2025MedAgent-Pro: Towards Evidence-based Multi-modal Medical Diagnosis via Reasoning Agentic Workflowcited by 73
  7. 2023Igniting Language Intelligence: The Hitchhiker's Guide From Chain-of-Thought Reasoning to Language Agentscited by 72
  8. 2024Large Language Model-based Human-Agent Collaboration for Complex Task Solvingcited by 38
  9. 2025GUARDIAN: Safeguarding LLM Multi-Agent Collaborations with Temporal Graph Modelingcited by 36
  10. 2023Interactive Natural Language Processingcited by 31
  11. 2025LLM-Based Human-Agent Collaboration and Interaction Systems: A Surveycited by 29
  12. 2025SurgRAW: Multi-Agent Workflow with Chain of Thought Reasoning for Robotic Surgical Video Analysiscited by 15
20 more
  1. 2025GuideBench: Benchmarking Domain-Oriented Guideline Following for LLM Agentscited by 14
  2. 2023Transforming Human-Centered AI Collaboration: Redefining Embodied Agents Capabilities through Interactive Grounded Language Instructionscited by 13
  3. 2024Computational Experiments Meet Large Language Model Based Agents: A Survey and Perspectivecited by 11
  4. 2024Causal Agent based on Large Language Modelcited by 10
  5. 2025AssistanceZero: Scalably Solving Assistance Gamescited by 10
  6. 2024Text2BIM: Generating Building Models Using a Large Language Model-based Multi-Agent Frameworkcited by 9
  7. 2025A Call for Collaborative Intelligence: Why Human-Agent Systems Should Precede AI Autonomycited by 9
  8. 2024Large Language Models as Minecraft Agentscited by 7
  9. 2024Teaching Embodied Reinforcement Learning Agents: Informativeness and Diversity of Language Usecited by 7
  10. 2024Towards a copilot in BIM authoring tool using a large language model-based agent for intelligent human-machine interactioncited by 6
  11. 2025How to Enable Effective Cooperation Between Humans and NLP Models: A Survey of Principles, Formalizations, and Beyondcited by 4
  12. 2025MotiveBench: How Far Are We From Human-Like Motivational Reasoning in Large Language Models?cited by 4
  13. 2025Preventing Rogue Agents Improves Multi-Agent Collaborationcited by 3
  14. 2025Carbon and Silicon, Coexist or Compete? A Survey on Human-AI Interactions in Agent-based Modeling and Simulationcited by 3
  15. 2026Intelli-Planner: Towards Customized Urban Planning via Large Language Model Empowered Reinforcement Learningcited by 3
  16. 2025Divide by Question, Conquer by Agent: SPLIT-RAG with Question-Driven Graph Partitioningcited by 2
  17. 2024IDAT: A Multi-Modal Dataset and Toolkit for Building and Evaluating Interactive Task-Solving Agentscited by 1
  18. 2026Before Agents Speak: Pre-hoc Failure Risk Inference in Multi-Agent Systemscited by 1
  19. 2025COCORELI: Enforcing Execution Preconditions for Reliable Collaborative Instruction Followingcited by 0
  20. 2026A Baseline Assessment for the Agentic Web Technology Stackcited by 0