Learning from Relevant Subgoals in Successful Dialogs using Iterative Training for Task-oriented Dialog Systems
Not on the map. Why?
Abstract (arXiv)
Task-oriented Dialog (ToD) systems have to solve multiple subgoals to accomplish user goals, whereas feedback is often obtained only at the end of the dialog. In this work, we propose SUIT (SUbgoal-aware ITerative Training), an iterative training approach for improving ToD systems. We sample dialogs from the model we aim to improve and determine subgoals that contribute to dialog success using distant supervision to obtain high quality training samples. We show how this data improves supervised fine-tuning or, alternatively, preference learning results. SUIT is able to iteratively generate more data instead of relying on fixed static sets. SUIT reaches new state-of-the-art performance on a popular ToD benchmark.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 22 of its 32 references that are in the map. First posted on arXiv 25 Nov 2024.
Builds on
22 of 32 references.
References · 22
- 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
- 2017Deep reinforcement learning from human preferencescited by 3,912
- 2020Learning to summarize from human feedbackcited by 2,588
- 2023A General Theoretical Paradigm to Understand Learning from Human Preferencescited by 912
- 2020A Simple Language Model for Task-Oriented Dialoguecited by 332
- 2023Preference Ranking Optimization for Human Alignmentcited by 285
- 2023A Survey of Reinforcement Learning from Human Feedbackcited by 264
- 2024Direct Language Model Alignment from Online AI Feedbackcited by 260
- 2020MultiWOZ 2.2 : A Dialogue Dataset with Additional Annotation Corrections and State Tracking Baselinescited by 204
- 2023Some things are more CRINGE than others: Iterative Preference Optimization with the Pairwise Cringe Losscited by 108
- 2021GALAXY: A Generative Pre-trained Model for Task-Oriented Dialog with Semi-Supervised Learning and Explicit Policy Injectioncited by 102
- 2021Dialogue State Tracking with a Language Model using Schema-Driven Promptingcited by 94
10 more
- 2019Rethinking Action Spaces for Reinforcement Learning in End-to-end Dialog Agents with Latent Variable Modelscited by 53
- 2022Description-Driven Task-Oriented Dialog Modelingcited by 50
- 2021Shades of BLEU, Flavours of Success: The Case of MultiWOZcited by 48
- 2022The CRINGE Loss: Learning what language not to modelcited by 39
- 2020LAVA: Latent Action Spaces via Variational Auto-encoding for Dialogue Policy Optimizationcited by 30
- 2023Task-Optimized Adapters for an End-to-End Task-Oriented Dialogue Systemcited by 30
- 2021Amendable Generation for Dialogue State Trackingcited by 21
- 2023Fantastic Rewards and How to Tame Them: A Case Study on Reward Learning for Task-oriented Dialogue Systemscited by 19
- 2022Mars: Modeling Context & State Representations with Contrastive Learning for End-to-End Task-Oriented Dialogcited by 11
- 2022Taming Continuous Posteriors for Latent Variational Dialogue Policiescited by 1
Built on by
Citing papers per year: 2025: 1.
1 paper.