mlsanityData to 7 Oct 2026

Continual Learning for Grounded Instruction Generation by Observing Human Following Behavior

arXiv 2108.04812 ↗First posted 10 Aug 2021cs.CL cs.AI cs.LGcited by 19

Not on the map. Why?

Abstract (arXiv)

We study continual learning for natural language instruction generation, by observing human users' instruction execution. We focus on a collaborative scenario, where the system both acts and delegates tasks to human users using natural language. We compare user execution of generated instructions to the original system intent as an indication to the system's success communicating its intent. We show how to use this signal to improve the system's ability to generate instructions via contextual bandit learning. In interaction with real users, our system demonstrates dramatic improvements in its ability to generate language over time.

Key facts. Cited by 19 papers in the map (data through 7 Oct 2026). Builds on 31 of its 58 references that are in the map. First posted on arXiv 10 Aug 2021.

Builds on

31 of 58 references.

References · 31

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  3. 2017Deep reinforcement learning from human preferencescited by 3,846
  4. 2014Sequence to Sequence Learning with Neural Networkscited by 2,974
  5. 2020Learning to summarize from human feedbackcited by 2,562
  6. 2016iCaRL: Incremental Classifier and Representation Learningcited by 2,313
  7. 2019Learning to Navigate Unseen Environments: Back Translation with Environmental Dropoutcited by 241
  8. 2017Survey of the State of the Art in Natural Language Generation: Core tasks, applications and evaluationcited by 206
  9. 2017Deal or No Deal? End-to-End Learning for Negotiation Dialoguescited by 202
  10. 2017Interactive Learning from Policy-Dependent Human Feedbackcited by 155
  11. 2017Deep TAMER: Interactive Agent Shaping in High-Dimensional State Spacescited by 122
  12. 2016Optimal and Adaptive Off-policy Evaluation in Contextual Banditscited by 111
19 more
  1. 2020Human-centric Dialog Training via Offline Reinforcement Learningcited by 92
  2. 2017Reinforcement Learning for Bandit Neural Machine Translation with Simulated Human Feedbackcited by 71
  3. 2018Can Neural Machine Translation be Improved with User Feedback?cited by 71
  4. 2019Executing Instructions in Situated Collaborative Interactionscited by 68
  5. 2018Reliability and Learnability of Human Bandit Feedback for Sequence-to-Sequence Reinforcement Learningcited by 57
  6. 2019Deep Reinforcement Learning from Policy-Dependent Human Feedbackcited by 56
  7. 2017Unified Pragmatic Models for Generating and Following Instructionscited by 55
  8. 2018Dialogue Learning with Human Teaching and Feedback in End-to-End Trainable Task-Oriented Dialogue Systemscited by 53
  9. 2021On the Evaluation of Vision-and-Language Navigation Instructionscited by 45
  10. 2016Learning Language Games through Interactioncited by 43
  11. 2019Learning to Map Natural Language Instructions to Physical Quadcopter Control using Simulated Flightcited by 43
  12. 2019Characterizing the dynamics of learning in repeated reference gamescited by 34
  13. 2018Improving a Neural Semantic Parser by Counterfactual Learning from Human Bandit Feedbackcited by 32
  14. 2017Bandit Structured Prediction for Neural Sequence-to-Sequence Learningcited by 31
  15. 2019Continual adaptation for efficient machine communicationcited by 28
  16. 2018HexaConvcited by 27
  17. 2019Encoder-Agnostic Adaptation for Conditional Language Generationcited by 27
  18. 2017Counterfactual Learning from Bandit Feedback under Deterministic Logging: A Case Study in Statistical Machine Translationcited by 13
  19. 2016Simultaneous Control and Human Feedback in the Training of a Robotic Agent with Actor-Critic Reinforcement Learningcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 6; 2023: 4; 2024: 5; 2025: 1; 2026: 2.

19 papers.

Sort:

By area:

  1. 2023Personalized Soups: Personalized Large Language Model Alignment via Post-hoc Parameter Mergingcited by 271
  2. 2021Less is More: Generating Grounded Navigation Instructions from Landmarkscited by 62
  3. 2022Continual Learning for Instruction Following from Realtime Feedbackcited by 21
  4. 2022Simulating Bandit Learning from User Feedback for Extractive Question Answeringcited by 14
  5. 2022Pragmatics in Language Grounding: Phenomena, Tasks, and Modeling Approachescited by 12
  6. 2024Grounding Language in Multi-Perspective Referential Communicationcited by 12
  7. 2023Continually Improving Extractive QA via Human Feedbackcited by 11
  8. 2022Collecting Interactive Multi-modal Datasets for Grounded Language Understandingcited by 10
  9. 2022MUG: Interactive Multimodal Grounding on User Interfacescited by 9
  10. 2023Dealing with Semantic Underspecification in Multimodal NLPcited by 9
  11. 2024Infer Human's Intentions Before Following Natural Language Instructionscited by 9
  12. 2024CoGen: Learning from Feedback with Coupled Comprehension and Generationcited by 7
7 more
  1. 2024Retrospective Learning from Interactionscited by 6
  2. 2022Many Episode Learning in a Modular Embodied Agent via End-to-End Interactioncited by 1
  3. 2023CB2: Collaborative Natural Language Interaction Research Platformcited by 1
  4. 2024Improving Dialogue Agents by Decomposing One Global Explicit Annotation with Local Implicit Multimodal Feedbackcited by 1
  5. 2025Mixed-Initiative Dialog for Human-Robot Collaborative Manipulationcited by 1
  6. 2026Efficient Test-Time Adaptation through Human-AI Interactioncited by 0
  7. 2026Learning to Refer from Estimated Listener Gazecited by 0