mlsanityData to 7 Oct 2026

A Framework for Learning to Request Rich and Contextually Useful Information from Humans

arXiv 2110.08258 ↗First posted 14 Oct 2021cs.LG cs.AI cs.HCcited by 12

Not on the map. Why?

Abstract (arXiv)

When deployed, AI agents will encounter problems that are beyond their autonomous problem-solving capabilities. Leveraging human assistance can help agents overcome their inherent limitations and robustly cope with unfamiliar situations. We present a general interactive framework that enables an agent to request and interpret rich, contextually useful information from an assistant that has knowledge about the task and the environment. We demonstrate the practicality of our framework on a simulated human-assisted navigation problem. Aided with an assistance-requesting policy learned by our method, a navigation agent achieves up to a 7x improvement in success rate on tasks that take place in previously unseen environments, compared to fully autonomous behavior. We show that the agent can take advantage of different types of information depending on the context, and analyze the benefits and challenges of learning the assistance-requesting policy when the assistant can recursively decompose tasks into subtasks.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 52 of its 97 references that are in the map. First posted on arXiv 14 Oct 2021.

Builds on

52 of 97 references.

References · 52

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  4. 2017Deep reinforcement learning from human preferencescited by 3,846
  5. 2013Visualizing and Understanding Convolutional Networkscited by 2,278
  6. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,218
  7. 2017Understanding Black-box Predictions via Influence Functionscited by 1,714
  8. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,013
  9. 2019ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Taskscited by 773
  10. 2017Adversarial Examples for Evaluating Reading Comprehension Systemscited by 568
  11. 2018e-SNLI: Natural Language Inference with Natural Language Explanationscited by 468
  12. 2016Visual Dialogcited by 377
38 more
  1. 2019Explain Yourself! Leveraging Language Models for Commonsense Reasoningcited by 367
  2. 2019REVERIE: Remote Embodied Visual Referring Expression in Real Indoor Environmentscited by 330
  3. 2020A Recurrent Vision-and-Language BERT for Navigationcited by 300
  4. 2021PEBBLE: Feedback-Efficient Interactive Reinforcement Learning via Relabeling Experience and Unsupervised Pre-trainingcited by 298
  5. 2016Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivationcited by 282
  6. 2020Opportunities and Challenges in Explainable Artificial Intelligence (XAI): A Surveycited by 252
  7. 2019Vision-and-Dialog Navigationcited by 246
  8. 2021TEACh: Task-driven Embodied Agents that Chatcited by 194
  9. 2019Paired Open-Ended Trailblazer (POET): Endlessly Generating Increasingly Complex and Diverse Learning Environments and Their Solutionscited by 182
  10. 2016Generating Visual Explanationscited by 156
  11. 2019A Survey of Reinforcement Learning Informed by Natural Languagecited by 154
  12. 2019Language as an Abstraction for Hierarchical Deep Reinforcement Learningcited by 138
  13. 2021Goal-Conditioned Reinforcement Learning with Imagined Subgoalscited by 129
  14. 2016GuessWhat?! Visual object discovery through multi-modal dialoguecited by 112
  15. 2018Learning to Ask Good Questions: Ranking Clarification Questions using Neural Expected Value of Perfect Informationcited by 110
  16. 2018Mapping Instructions to Actions in 3D Environments with Visual Goal Predictioncited by 101
  17. 2017Grounded Language Learning in a Simulated 3D Worldcited by 100
  18. 2019Help, Anna! Visual Navigation with Natural Multimodal Assistance via Retrospective Curiosity-Encouraging Imitation Learningcited by 100
  19. 2020NILE : Natural Language Inference with Faithful Natural Language Explanationscited by 96
  20. 2018Hierarchical Imitation and Reinforcement Learningcited by 88
  21. 2017Learning how to Active Learn: A Deep Reinforcement Learning Approachcited by 79
  22. 2018Vision-based Navigation with Language-based Assistance via Imitation Learning with Indirect Interventioncited by 77
  23. 2016Reasoning About Pragmatics with Neural Listeners and Speakerscited by 68
  24. 2020Language as a Cognitive Tool to Imagine Goals in Curiosity-Driven Explorationcited by 68
  25. 2019From Language to Goals: Inverse Reinforcement Learning for Vision-Based Instruction Followingcited by 67
  26. 2021MindCraft: Theory of Mind Modeling for Situated Dialogue in Collaborative Taskscited by 62
  27. 2016Generating Natural Questions About an Imagecited by 56
  28. 2017Unified Pragmatic Models for Generating and Following Instructionscited by 55
  29. 2018Learning to Teach in Cooperative Multiagent Reinforcement Learningcited by 45
  30. 2018Talk the Walk: Navigating New York City through Grounded Dialoguecited by 44
  31. 2020Learning Rewards from Linguistic Feedbackcited by 41
  32. 2020Asking Questions the Human Way: Scalable Question-Answer Generation from Text Corpuscited by 40
  33. 2020How to Avoid Being Eaten by a Grue: Structured Exploration Strategies for Textual Worldscited by 38
  34. 2020RMM: A Recursive Mental Model for Dialog Navigationcited by 32
  35. 2021Few-shot Language Coordination by Modeling Theory of Mindcited by 32
  36. 2021Interactive Learning from Activity Descriptioncited by 26
  37. 2020Inverse Reinforcement Learning with Natural Language Goalscited by 20
  38. 2019ACTRCE: Augmenting Experience via Teacher's Advice For Multi-Goal Reinforcement Learningcited by 17

50 most cited of 52 All 52 (JSON)

Built on by

Cited by year

Citing papers per year: 2022: 3; 2023: 4; 2024: 5.

12 papers.

  1. 2023Cognitive Architectures for Language Agentscited by 422
  2. 2022Vision-and-Language Navigation: A Survey of Tasks, Methods, and Future Directionscited by 116
  3. 2022Retrospectives on the Embodied AI Workshopcited by 46
  4. 2023Think, Act, and Ask: Open-World Interactive Personalized Robot Navigationcited by 31
  5. 2024DriVLMe: Enhancing LLM-based Autonomous Driving Agents with Embodied and Social Experiencescited by 27
  6. 2024ReSpAct: Harmonizing Reasoning, Speaking, and Acting Towards Building Large Language Model-Based Conversational AI Agentscited by 14
  7. 2024Teaching Embodied Reinforcement Learning Agents: Informativeness and Diversity of Language Usecited by 7
  8. 2024Collaborative Instance Object Navigation: Leveraging Uncertainty-Awareness to Minimize Human-Agent Dialoguescited by 7
  9. 2022Good Time to Ask: A Learning Framework for Asking for Help in Embodied Visual Navigationcited by 3
  10. 2023RE-MOVE: An Adaptive Policy Design for Robotic Navigation Tasks in Dynamic Environments via Language-Based Feedbackcited by 3
  11. 2023Progressively Efficient Learningcited by 1
  12. 2024IDAT: A Multi-Modal Dataset and Toolkit for Building and Evaluating Interactive Task-Solving Agentscited by 1