mlsanityData to 14 Oct 2026

Context-Aware Language Modeling for Goal-Oriented Dialogue Systems

arXiv 2204.10198 ↗First posted 18 Apr 2022cs.CL cs.AIcited by 22

Not on the map. Why?

Abstract (arXiv)

Goal-oriented dialogue systems face a trade-off between fluent language generation and task-specific control. While supervised learning with large language models is capable of producing realistic text, how to steer such responses towards completing a specific task without sacrificing language quality remains an open question. In this work, we formulate goal-oriented dialogue as a partially observed Markov decision process, interpreting the language model as a representation of both the dynamics and the policy. This view allows us to extend techniques from learning-based control, such as task relabeling, to derive a simple and effective method to finetune language models in a goal-aware way, leading to significantly improved task performance. We additionally introduce a number of training strategies that serve to better focus the model on the task at hand. We evaluate our method, Context-Aware Language Models (CALM), on a practical flight-booking task using AirDialogue. Empirically, CALM outperforms the state-of-the-art method by 7% in terms of task success, matching human-level task performance.

Key facts. Cited by 22 papers in the map (data through 14 Oct 2026). Builds on 38 of its 66 references that are in the map. First posted on arXiv 18 Apr 2022.

Builds on

38 of 66 references.

References · 38

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  5. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,920
  6. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,293
  7. 2018Off-Policy Deep Reinforcement Learning without Explorationcited by 1,241
  8. 2017Hindsight Experience Replaycited by 1,136
  9. 2018MultiWOZ -- A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modellingcited by 766
  10. 2019Behavior Regularized Offline Reinforcement Learningcited by 662
  11. 2020Towards a Human-like Open-Domain Chatbotcited by 470
  12. 2020A Simple Language Model for Task-Oriented Dialoguecited by 332
26 more
  1. 2019Learning Latent Plans from Playcited by 316
  2. 2016Deep Reinforcement Learning for Dialogue Generationcited by 288
  3. 2019Way Off-Policy Batch Deep Reinforcement Learning of Implicit Human Preferences in Dialogcited by 284
  4. 2021TAPEX: Table Pre-training via Learning a Neural SQL Executorcited by 235
  5. 2017Deal or No Deal? End-to-End Learning for Negotiation Dialoguescited by 209
  6. 2018Semi-parametric Topological Memory for Navigationcited by 209
  7. 2018Decoupling Strategy and Generation in Negotiation Dialoguescited by 162
  8. 2019Task-Oriented Dialog Systems that Consider Multiple Appropriate Responses under the Same Contextcited by 99
  9. 2018Temporal Difference Models: Model-Free Deep RL for Model-Based Controlcited by 89
  10. 2022When Should We Prefer Offline Reinforcement Learning Over Behavioral Cloning?cited by 89
  11. 2020SOLOIST: Building Task Bots at Scale with Transfer Learning and Machine Teachingcited by 71
  12. 2020Rewriting History with Inverse RL: Hindsight Inference for Policy Improvementcited by 60
  13. 2018Dialogue Learning with Human Teaching and Feedback in End-to-End Trainable Task-Oriented Dialogue Systemscited by 53
  14. 2019Rethinking Action Spaces for Reinforcement Learning in End-to-end Dialog Agents with Latent Variable Modelscited by 53
  15. 2019Semantically Conditioned Dialog Response Generation via Hierarchical Disentangled Self-Attentioncited by 44
  16. 2019Recommendation as a Communication Game: Self-Supervised Bot-Play for Goal-oriented Dialoguecited by 44
  17. 2016A Sequence-to-Sequence Model for User Simulation in Spoken Dialogue Systemscited by 39
  18. 2020Modelling Hierarchical Structure between Dialogue Policy and Natural Language Generator with Option Framework for Task-oriented Dialogue Systemcited by 28
  19. 2016Sequence-to-Sequence Generation for Spoken Dialogue via Deep Syntax Trees and Stringscited by 27
  20. 2019Learning to Reach Goals via Iterated Supervised Learningcited by 26
  21. 2019Alternating Recurrent Dialog Model with Large-scale Pre-trained Language Modelscited by 25
  22. 2017Sample-efficient Actor-Critic Reinforcement Learning with Supervised Data for Dialogue Managementcited by 18
  23. 2021Towards Automatic Evaluation of Dialog Systems: A Model-Free Off-Policy Evaluation Approachcited by 16
  24. 2017A Copy-Augmented Sequence-to-Sequence Architecture Gives Good Performance on Task-Oriented Dialoguecited by 14
  25. 2016Continuously Learning Neural Dialogue Managementcited by 12
  26. 2016Coherent Dialogue with Attention-based Language Modelscited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 3; 2023: 11; 2024: 2; 2025: 5; 2026: 1.

22 papers.

Sort:

By area:

  1. 2023Language to Rewards for Robotic Skill Synthesiscited by 312
  2. 2023Foundation Models for Decision Making: Problems, Methods, and Opportunitiescited by 164
  3. 2022Offline RL for Natural Language Generation with Implicit Language Q Learningcited by 138
  4. 2023LMRL Gym: Benchmarks for Multi-Turn Reinforcement Learning with Language Modelscited by 88
  5. 2023Grounded Decoding: Guiding Text Generation with Grounded Models for Embodied Agentscited by 67
  6. 2023Leveraging Large Language Models in Conversational Recommender Systemscited by 63
  7. 2023A Survey on Transformers in Reinforcement Learningcited by 50
  8. 2022On Transforming Reinforcement Learning by Transformer: The Development Trajectorycited by 36
  9. 2023Fantastic Rewards and How to Tame Them: A Case Study on Reward Learning for Task-oriented Dialogue Systemscited by 19
  10. 2025Large language models for artificial general intelligence (AGI): A survey of foundational principles and approachescited by 18
  11. 2025Evaluating the Goal-Directedness of Large Language Modelscited by 16
  12. 2023Building Persona Consistent Dialogue Agents with Offline Reinforcement Learningcited by 14
10 more
  1. 2025Reinforcement Learning for Machine Learning Engineering Agentscited by 14
  2. 2023Asking Before Acting: Gather Information in Embodied Decision Making with Language Modelscited by 12
  3. 2024Q-SFT: Q-Learning for Language Models via Supervised Fine-Tuningcited by 10
  4. 2023Enhancing Large Language Model Induced Task-Oriented Dialogue Systems Through Look-Forward Motivated Goalscited by 8
  5. 2024Target-constrained Bidirectional Planning for Generation of Target-oriented Proactive Dialoguecited by 6
  6. 2025Learning from Failures in Multi-Attempt Reinforcement Learningcited by 6
  7. 2023The Inner Sentiments of a Thoughtcited by 4
  8. 2026Hallucination Detection and Mitigation in Large Language Modelscited by 4
  9. 2022Dual Generator Offline Reinforcement Learningcited by 1
  10. 2025Prompt Optimization with Logged Bandit Datacited by 1