mlsanityData to 13 Oct 2026

Dialogue Learning With Human-In-The-Loop

arXiv 1611.09823 ↗First posted 29 Nov 2016cs.AI cs.CLcited by 46

Not on the map. Why?

Abstract (arXiv)

An important aspect of developing conversational agents is to give a bot the ability to improve through communicating with humans and to learn from the mistakes that it makes. Most research has focused on learning from fixed training sets of labeled data rather than interacting with a dialogue partner in an online fashion. In this paper we explore this direction in a reinforcement learning setting where the bot improves its question-answering ability from feedback a teacher gives following its generated responses. We build a simulator that tests various aspects of such learning in a synthetic environment, and introduce models that work in this regime. Finally, real experiments with Mechanical Turk validate the approach.

Key facts. Cited by 46 papers in the map (data through 13 Oct 2026). First posted on arXiv 29 Nov 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2018: 1; 2019: 2; 2020: 2; 2021: 8; 2022: 10; 2023: 11; 2024: 7; 2025: 2; 2026: 3.

46 papers.

Sort:

By area:

  1. 2019Fine-Tuning Language Models from Human Preferencescited by 2,139
  2. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,433
  3. 2023AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedbackcited by 745
  4. 2019Way Off-Policy Batch Deep Reinforcement Learning of Implicit Human Preferences in Dialogcited by 284
  5. 2021A Survey of Human-in-the-loop for Machine Learningcited by 214
  6. 2022BlenderBot 3: a deployed conversational agent that continually learns to responsibly engagecited by 198
  7. 2021Recent Advances in Deep Learning Based Dialogue Systems: A Systematic Surveycited by 130
  8. 2023Training Language Models with Language Feedback at Scalecited by 126
  9. 2022A Survey of Deep Learning for Mathematical Reasoningcited by 124
  10. 2020Human-centric Dialog Training via Offline Reinforcement Learningcited by 92
  11. 2024RLHF Deciphered: A Critical Analysis of Reinforcement Learning from Human Feedback for LLMscited by 90
  12. 2022A data-driven approach for learning to control computerscited by 87
34 more
  1. 2024Aligning LLM Agents by Learning Latent Preference from User Editscited by 66
  2. 2021Building and Evaluating Open-Domain Dialogue Corpora with Clarifying Questionscited by 63
  3. 2023Bridging the Gap: A Survey on Integrating (Human) Feedback for Natural Language Generationcited by 55
  4. 2023The Past, Present and Better Future of Feedback Learning in Large Language Models for Subjective Human Preferences and Valuescited by 54
  5. 2022Training Language Models with Language Feedbackcited by 49
  6. 2026Expanding the Capabilities of Reinforcement Learning via Text Feedbackcited by 49
  7. 2020Learning Rewards from Linguistic Feedbackcited by 41
  8. 2021Putting Humans in the Natural Language Processing Loop: A Surveycited by 40
  9. 2022Learning New Skills after Deployment: Improving open-domain internet-driven dialogue with human feedbackcited by 36
  10. 2022Using Interactive Feedback to Improve the Accuracy and Explainability of Question Answering Systems Post-Deploymentcited by 25
  11. 2023LeTI: Learning to Generate from Textual Interactionscited by 25
  12. 2018Deep Dyna-Q: Integrating Planning for Task-Completion Dialogue Policy Learningcited by 21
  13. 2025Multi-Agent Autonomous Driving Systems with Large Language Models: A Survey of Recent Advancescited by 21
  14. 2022When Life Gives You Lemons, Make Cherryade: Converting Feedback from Bad Responses into Good Labelscited by 20
  15. 2023DUnE: Dataset for Unified Editingcited by 19
  16. 2024RLVF: Learning from Verbal Feedback without Overgeneralizationcited by 18
  17. 2023Reasons to Reject? Aligning Language Models with Judgmentscited by 15
  18. 2025Alignment and Safety in Large Language Models: Safety Mechanisms, Training Paradigms, and Emerging Challengescited by 15
  19. 2023Leveraging Implicit Feedback from Deployment Data in Dialoguecited by 14
  20. 2022Learning from data in the mixed adversarial non-adversarial case: Finding the helpers and ignoring the trollscited by 11
  21. 2024Concept Alignmentcited by 10
  22. 2023Let Me Teach You: Pedagogical Foundations of Feedback for Language Modelscited by 9
  23. 2024Retrospective Learning from Interactionscited by 6
  24. 2024It Takes Two: On the Seamlessness between Reward and Policy Model in RLHFcited by 5
  25. 2023System-Level Natural Language Feedbackcited by 4
  26. 2021Potential Impacts of Smart Homes on Human Behavior: A Reinforcement Learning Approachcited by 1
  27. 2022CausalDialogue: Modeling Utterance-level Causality in Conversationscited by 1
  28. 2024A Probabilistic Approach for Model Alignment with Human Comparisonscited by 1
  29. 2021Topic-time Heatmaps for Human-in-the-loop Topic Detection and Trackingcited by 0
  30. 2021Findings from Experiments of On-line Joint Reinforcement Learning of Semantic Parser and Dialogue Manager with real Userscited by 0
  31. 2021Improving mathematical questioning in teacher trainingcited by 0
  32. 2022A Reinforcement Learning-based Offensive semantics Censorship System for Chatbotscited by 0
  33. 2026From Automation to Collaboration: Human-in-the-Loop Methods for Safe and Trustworthy NLPcited by 0
  34. 2026Shockingly Simple Self-retrospection Improves Agentic Models Without RLcited by 0