mlsanityData to 7 Oct 2026

Interactive Learning from Activity Description

arXiv 2102.07024 ↗First posted 13 Feb 2021cs.CL cs.AI cs.HCcited by 26

Not on the map. Why?

Abstract (arXiv)

We present a novel interactive learning protocol that enables training request-fulfilling agents by verbally describing their activities. Unlike imitation learning (IL), our protocol allows the teaching agent to provide feedback in a language that is most appropriate for them. Compared with reward in reinforcement learning (RL), the description feedback is richer and allows for improved sample complexity. We develop a probabilistic framework and an algorithm that practically implements our protocol. Empirical results in two challenging request-fulfilling problems demonstrate the strengths of our approach: compared with RL baselines, it is more sample-efficient; compared with IL baselines, it achieves competitive success rates without requiring the teaching agent to be able to demonstrate the desired behavior using the learning agent's actions. Apart from empirical evaluation, we also provide theoretical guarantees for our algorithm under certain assumptions about the teacher and the environment.

Key facts. Cited by 26 papers in the map (data through 7 Oct 2026). Builds on 43 of its 72 references that are in the map. First posted on arXiv 13 Feb 2021.

Builds on

43 of 72 references.

References · 43

  1. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,218
  2. 2017Hindsight Experience Replaycited by 1,096
  3. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,013
  4. 2016Learning to Communicate with Deep Multi-Agent Reinforcement Learningcited by 561
  5. 2018Speaker-Follower Models for Vision-and-Language Navigationcited by 337
  6. 2019Learning to Navigate Unseen Environments: Back Translation with Environmental Dropoutcited by 241
  7. 2018BabyAI: A Platform to Study the Sample Efficiency of Grounded Language Learningcited by 238
  8. 2018Touchdown: Natural Language Navigation and Spatial Reasoning in Visual Street Environmentscited by 238
  9. 2020FLAMBE: Structural Complexity and Representation Learning of Low Rank MDPscited by 232
  10. 2016Multi-Agent Cooperation and the Emergence of (Natural) Languagecited by 181
  11. 2019Language as an Abstraction for Hierarchical Deep Reinforcement Learningcited by 138
  12. 2017Emergence of Language with Multi-agent Games: Learning to Communicate with Sequences of Symbolscited by 123
31 more
  1. 2019Using Natural Language for Reward Shaping in Reinforcement Learningcited by 114
  2. 2017Third-Person Imitation Learningcited by 103
  3. 2019General Evaluation for Instruction Conditioned Navigation using Dynamic Time Warpingcited by 102
  4. 2017Grounded Language Learning in a Simulated 3D Worldcited by 100
  5. 2019Help, Anna! Visual Navigation with Natural Multimodal Assistance via Retrospective Curiosity-Encouraging Imitation Learningcited by 100
  6. 2017Natural Language Does Not Emerge 'Naturally' in Multi-Agent Dialogcited by 86
  7. 2017Mapping Instructions and Visual Observations to Actions with Reinforcement Learningcited by 80
  8. 2019Provably Efficient Imitation Learning from Observation Alonecited by 78
  9. 2018Vision-based Navigation with Language-based Assistance via Imitation Learning with Indirect Interventioncited by 77
  10. 2017Learning with Latent Languagecited by 75
  11. 2017Gated-Attention Architectures for Task-Oriented Language Groundingcited by 72
  12. 2017Reinforcement Learning for Bandit Neural Machine Translation with Simulated Human Feedbackcited by 71
  13. 2017Learning Cooperative Visual Dialog Agents with Deep Reinforcement Learningcited by 70
  14. 2016Reasoning About Pragmatics with Neural Listeners and Speakerscited by 68
  15. 2020Language as a Cognitive Tool to Imagine Goals in Curiosity-Driven Explorationcited by 68
  16. 2019From Language to Goals: Inverse Reinforcement Learning for Vision-Based Instruction Followingcited by 67
  17. 2018Reliability and Learnability of Human Bandit Feedback for Sequence-to-Sequence Reinforcement Learningcited by 57
  18. 2019Reinforcement Learning with Convex Constraintscited by 57
  19. 2017Unified Pragmatic Models for Generating and Following Instructionscited by 55
  20. 2015Listen, Attend, and Walk: Neural Mapping of Navigational Instructions to Action Sequencescited by 50
  21. 2016Learning Language Games through Interactioncited by 43
  22. 2020Learning Rewards from Linguistic Feedbackcited by 41
  23. 2017Bandit Structured Prediction for Neural Sequence-to-Sequence Learningcited by 31
  24. 2020An Imitation Game for Learning Semantic Parsers from User Interactioncited by 23
  25. 2020Inverse Reinforcement Learning with Natural Language Goalscited by 20
  26. 2019ACTRCE: Augmenting Experience via Teacher's Advice For Multi-Goal Reinforcement Learningcited by 17
  27. 2015Learning in the Rational Speech Acts Modelcited by 16
  28. 2017Teaching Machines to Describe Images via Natural Language Feedbackcited by 8
  29. 2019Learning to Learn Semantic Parsers from Natural Language Supervisioncited by 7
  30. 2019Pre-Learning Environment Representations for Data-Efficient Neural Instruction Followingcited by 6
  31. 2020Towards Effective Human-AI Collaboration in GUI-Based Interactive Task Learning Agentscited by 2

Built on by

Cited by year

Citing papers per year: 2021: 5; 2022: 7; 2023: 10; 2024: 4.

26 papers.

Sort:

By area:

  1. 2023CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquingcited by 751
  2. 2023Cognitive Architectures for Language Agentscited by 422
  3. 2023REFINER: Reasoning Feedback on Intermediate Representationscited by 203
  4. 2023Training Language Models with Language Feedback at Scalecited by 125
  5. 2024RLHF Deciphered: A Critical Analysis of Reinforcement Learning from Human Feedback for LLMscited by 89
  6. 2024Aligning LLM Agents by Learning Latent Preference from User Editscited by 66
  7. 2022Training Language Models with Language Feedbackcited by 48
  8. 2023Distilling Internet-Scale Vision-Language Models into Embodied Agentscited by 30
  9. 2023LLF-Bench: Benchmark for Interactive Learning from Language Feedbackcited by 30
  10. 2022How to talk so AI will learn: Instructions, descriptions, and autonomycited by 22
  11. 2023Passive learning of active causal strategies in agents and language modelscited by 21
  12. 2022Imitating Past Successes can be Very Suboptimalcited by 20
14 more
  1. 2022Language and Culture Internalisation for Human-Like Autotelic AIcited by 14
  2. 2021A Framework for Learning to Request Rich and Contextually Useful Information from Humanscited by 12
  3. 2024Language-Guided World Models: A Model-Based Approach to AI Controlcited by 12
  4. 2023Language Models are Bounded Pragmatic Speakers: Understanding RLHF from a Bayesian Cognitive Modeling Perspectivecited by 10
  5. 2021Grounding Spatio-Temporal Language with Transformerscited by 7
  6. 2021SocialAI: Benchmarking Socio-Cognitive Abilities in Deep Reinforcement Learning Agentscited by 7
  7. 2022Interaction-Grounded Learning with Action-inclusive Feedbackcited by 7
  8. 2023Learning to Summarize and Answer Questions about a Virtual Robot's Past Actionscited by 5
  9. 2021Learning to Guide and to Be Guided in the Architect-Builder Problemcited by 3
  10. 2022Grounding Hindsight Instructions in Multi-Goal Reinforcement Learning for Roboticscited by 3
  11. 2021Towards Teachable Autotelic Agentscited by 2
  12. 2022Teachable Reinforcement Learning via Advice Distillationcited by 2
  13. 2023Progressively Efficient Learningcited by 1
  14. 2024Provable Interactive Learning with Hindsight Instruction Feedbackcited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2019Learning to Interactively Learn and Assist
  2. 2021PEBBLE: Feedback-Efficient Interactive Reinforcement Learning via Relabeling Experience and Unsupervised Pre-training
  3. 2021Zero-shot Task Adaptation using Natural Language
  4. 2018Guiding Policies with Language via Meta-Learning
  5. 2024A Survey of Imitation Learning Methods, Environments and Metrics
  6. 2020Interactive Imitation Learning in State-Space
  7. 2025Structured Imitation Learning of Interactive Policies through Inverse Games
  8. 2024The intrinsic motivation of reinforcement and imitation learning for sequential tasks