mlsanityData to 7 Oct 2026

Composite Task-Completion Dialogue Policy Learning via Hierarchical Deep Reinforcement Learning

arXiv 1704.03084 ↗First posted 10 Apr 2017cs.CL cs.AI cs.LGcited by 44

Not on the map. Why?

Abstract (arXiv)

Building a dialogue agent to fulfill complex tasks, such as travel planning, is challenging because the agent has to learn to collectively complete multiple subtasks. For example, the agent needs to reserve a hotel and book a flight so that there leaves enough time for commute between arrival and hotel check-in. This paper addresses this challenge by formulating the task in the mathematical framework of options over Markov Decision Processes (MDPs), and proposing a hierarchical deep reinforcement learning approach to learning a dialogue manager that operates at different temporal scales. The dialogue manager consists of: (1) a top-level dialogue policy that selects among subtasks or options, (2) a low-level dialogue policy that selects primitive actions to complete the subtask given by the top-level policy, and (3) a global state tracker that helps ensure all cross-subtask constraints be satisfied. Experiments on a travel planning task with simulated and real users show that our approach leads to significant improvements over three baselines, two based on handcrafted rules and the other based on flat deep reinforcement learning.

Key facts. Cited by 44 papers in the map (data through 7 Oct 2026). First posted on arXiv 10 Apr 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 13; 2022: 10; 2023: 7; 2024: 7; 2025: 3; 2026: 4.

44 papers.

Sort:

By area:

  1. 2021Recent Advances in Deep Learning Based Dialogue Systems: A Systematic Surveycited by 130
  2. 2024Toward Self-Improvement of LLMs via Imagination, Searching, and Criticizingcited by 123
  3. 2021The Perils of Using Mechanical Turk to Evaluate Open-Ended Text Generationcited by 96
  4. 2022DialFRED: Dialogue-Enabled Agents for Embodied Instruction Followingcited by 63
  5. 2022A Survey on Recent Advances and Challenges in Reinforcement Learning Methods for Task-Oriented Dialogue Policy Learningcited by 30
  6. 2023Fantastic Rewards and How to Tame Them: A Case Study on Reward Learning for Task-oriented Dialogue Systemscited by 19
  7. 2021A Survey on Deep Reinforcement Learning for Audio-Based Applicationscited by 17
  8. 2022A Slot Is Not Built in One Utterance: Spoken Language Dialogs with Sub-Slotscited by 13
  9. 2021Wizard of Search Engine: Access to Information Through Conversations with Search Enginescited by 11
  10. 2021Generalization in Text-based Games via Hierarchical Reinforcement Learningcited by 11
  11. 2022KETOD: Knowledge-Enriched Task-Oriented Dialoguecited by 11
  12. 2021SelectAugment: Hierarchical Deterministic Sample Selection for Data Augmentationcited by 10
32 more
  1. 2023Reinforcement Learning for Generative AI: State of the Art, Opportunities and Open Research Challengescited by 10
  2. 2022Toward Self-learning End-to-End Task-Oriented Dialog Systemscited by 9
  3. 2025Krutrim LLM: Multilingual Foundational Model for over a Billion Peoplecited by 9
  4. 2021Imperfect also Deserves Reward: Multi-Level and Sequential Reward Modeling for Better Dialog Managementcited by 6
  5. 2021Teaching Models new APIs: Domain-Agnostic Simulators for Task Oriented Dialoguecited by 6
  6. 2022Evaluation of Synthetic Datasets for Conversational Recommender Systemscited by 6
  7. 2021NaRLE: Natural Language Models using Reinforcement Learning with Emotion Feedbackcited by 5
  8. 2021Integrating Pretrained Language Model for Dialogue Policy Learningcited by 5
  9. 2024On LLM Wizards: Identifying Large Language Models' Behaviors for Wizard of Oz Experimentscited by 4
  10. 2024TransferTOD: A Generalizable Chinese Multi-Domain Task-Oriented Dialogue System with Transfer Capabilitiescited by 4
  11. 2026Measuring and Mitigating the Distributional Gap Between Real and Simulated User Behaviorscited by 4
  12. 2021SYNERGY: Building Task Bots at Scale Using Symbolic Knowledge and Machine Teachingcited by 3
  13. 2024Rewarding What Matters: Step-by-Step Reinforcement Learning for Task-Oriented Dialoguecited by 3
  14. 2021WeaSuL: Weakly Supervised Dialogue Policy Learning: Reward Estimation for Multi-turn Dialoguecited by 2
  15. 2023Successor Features for Efficient Multisubject Controlled Text Generationcited by 2
  16. 2021Joint System-Wise Optimization for Pipeline Goal-Oriented Dialog Systemcited by 1
  17. 2022Trust in Language Grounding: a new AI challenge for human-robot teamscited by 1
  18. 2023A Review of Reinforcement Learning for Natural Language Processing, and Applications in Healthcarecited by 1
  19. 2024Unsupervised Human Preference Learningcited by 1
  20. 2022Recent Progress in Conversational AIcited by 0
  21. 2022Reinforcement Learning of Multi-Domain Dialog Policies Via Action Embeddingscited by 0
  22. 2022Improving a sequence-to-sequence nlp model using a reinforcement learning policy algorithmcited by 0
  23. 2023Why Guided Dialog Policy Learning performs well? Understanding the role of adversarial learning and its alternativecited by 0
  24. 2023JoTR: A Joint Transformer and Reinforcement Learning Framework for Dialog Policy Learningcited by 0
  25. 2023Dialog Action-Aware Transformer for Dialog Policy Learningcited by 0
  26. 2024Reinforcement Learning from Bagged Rewardcited by 0
  27. 2024Beyond Ontology in Dialogue State Tracking for Goal-Oriented Chatbotcited by 0
  28. 2025Policy Learning with a Natural Language Action Space: A Causal Approachcited by 0
  29. 2025DyBBT: Dynamic Balance via Bandit-inspired Targeting for Dialog Policy with Cognitive Dual-Systemscited by 0
  30. 2026Uncertainty as a Planning Signal: Multi-Turn Decision Making for Goal-Oriented Conversationcited by 0
  31. 2026DiPS: Dialogue Policy Selection for High-Stakes Persuasion Agentscited by 0
  32. 2026An Auditable Symbolic-RAG-Generative AI Architecture for Goal-Oriented Conversation Orchestrationcited by 0