# Precognition in Task-oriented Dialogue Understanding: Posterior Regularization by Future Context

arXiv: https://arxiv.org/abs/2203.03244 · HTML: https://mlsanity.com/paper/2203.03244/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 15 of its 25 references that are in the map.
- First posted on arXiv 7 Mar 2022.

## Builds on (most cited first, 15 of 15)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling](https://mlsanity.com/paper/1412.3555/) (2014)
- [Generating Sentences from a Continuous Space](https://mlsanity.com/paper/1511.06349/) (2015)
- [The Design and Implementation of XiaoIce, an Empathetic Social Chatbot](https://mlsanity.com/paper/1812.08989/) (2018)
- [Neural Variational Inference and Learning in Belief Networks](https://mlsanity.com/paper/1402.0030/) (2014)
- [Latent Alignment and Variational Attention](https://mlsanity.com/paper/1807.03756/) (2018)
- [SeaD: End-to-end Text-to-SQL Generation with Schema-aware Denoising](https://mlsanity.com/paper/2105.07911/) (2021)
- [AliMe Assist: An Intelligent Assistant for Creating an Innovative E-commerce Experience](https://mlsanity.com/paper/1801.05032/) (2018)
- [Regularizing Dialogue Generation by Imitating Implicit Scenarios](https://mlsanity.com/paper/2010.01893/) (2020)
- [CASA-NLU: Context-Aware Self-Attentive Natural Language Understanding for Task-Oriented Chatbots](https://mlsanity.com/paper/1909.08705/) (2019)
- [Conversational Machine Comprehension: a Literature Review](https://mlsanity.com/paper/2006.00671/) (2020)
- [NEXUS Network: Connecting the Preceding and the Following in Dialogue Generation](https://mlsanity.com/paper/1810.00671/) (2018)

## Abstract (arXiv)

Task-oriented dialogue systems have become overwhelmingly popular in recent researches. Dialogue understanding is widely used to comprehend users' intent, emotion and dialogue state in task-oriented dialogue systems. Most previous works on such discriminative tasks only models current query or historical conversations. Even if in some work the entire dialogue flow was modeled, it is not suitable for the real-world task-oriented conversations as the future contexts are not visible in such cases. In this paper, we propose to jointly model historical and future information through the posterior regularization method. More specifically, by modeling the current utterance and past contexts as prior, and the entire dialogue flow as posterior, we optimize the KL distance between these distributions to regularize our model during training. And only historical information is used for inference. Extensive experiments on two dialogue datasets validate the effectiveness of our proposed method, achieving superior results compared with all baseline models.