# Adversarial Advantage Actor-Critic Model for Task-Completion Dialogue Policy Learning

arXiv: https://arxiv.org/abs/1710.11277 · HTML: https://mlsanity.com/paper/1710.11277/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Oct 2017.

## Built on by (most cited first, 10 of 10)

- [Foundation Metrics for Evaluating Effectiveness of Healthcare Conversations Powered by Generative AI](https://mlsanity.com/paper/2309.12444/) (2023)
- [A Survey on Recent Advances and Challenges in Reinforcement Learning Methods for Task-Oriented Dialogue Policy Learning](https://mlsanity.com/paper/2202.13675/) (2022)
- [Deep Dyna-Q: Integrating Planning for Task-Completion Dialogue Policy Learning](https://mlsanity.com/paper/1801.06176/) (2018)
- [ReMeDi: Resources for Multi-domain, Multi-service, Medical Dialogues](https://mlsanity.com/paper/2109.00430/) (2021)
- [A Survey on Deep Reinforcement Learning for Audio-Based Applications](https://mlsanity.com/paper/2101.00240/) (2021)
- [Integrating Pretrained Language Model for Dialogue Policy Learning](https://mlsanity.com/paper/2111.01398/) (2021)
- [Enhancing PPO with Trajectory-Aware Hybrid Policies](https://mlsanity.com/paper/2502.15968/) (2025)
- [Rescue Conversations from Dead-ends: Efficient Exploration for Task-oriented Dialogue Policy Optimization](https://mlsanity.com/paper/2305.03262/) (2023)
- [TransferLight: Zero-Shot Traffic Signal Control on any Road-Network](https://mlsanity.com/paper/2412.09719/) (2024)
- [JoTR: A Joint Transformer and Reinforcement Learning Framework for Dialog Policy Learning](https://mlsanity.com/paper/2309.00230/) (2023)

## Abstract (arXiv)

This paper presents a new method --- adversarial advantage actor-critic (Adversarial A2C), which significantly improves the efficiency of dialogue policy learning in task-completion dialogue systems. Inspired by generative adversarial networks (GAN), we train a discriminator to differentiate responses/actions generated by dialogue agents from responses/actions by experts. Then, we incorporate the discriminator as another critic into the advantage actor-critic (A2C) framework, to encourage the dialogue agent to explore state-action within the regions where the agent takes actions similar to those of the experts. Experimental results in a movie-ticket booking domain show that the proposed Adversarial A2C can accelerate policy exploration efficiently.