# Incorporating Causal Analysis into Diversified and Logical Response Generation

arXiv: https://arxiv.org/abs/2209.09482 · HTML: https://mlsanity.com/paper/2209.09482/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 51 references that are in the map.
- First posted on arXiv 20 Sep 2022.

## Builds on (most cited first, 30 of 30)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [A Diversity-Promoting Objective Function for Neural Conversation Models](https://mlsanity.com/paper/1510.03055/) (2015)
- [Counterfactual VQA: A Cause-Effect Look at Language Bias](https://mlsanity.com/paper/2006.04315/) (2020)
- [Learning Discourse-level Diversity for Neural Dialog Models using Conditional Variational Autoencoders](https://mlsanity.com/paper/1703.10960/) (2017)
- [A Survey on Causal Inference](https://mlsanity.com/paper/2002.02770/) (2020)
- [A Neural Conversational Model](https://mlsanity.com/paper/1506.05869/) (2015)
- [A Meta-Transfer Objective for Learning to Disentangle Causal Mechanisms](https://mlsanity.com/paper/1901.10912/) (2019)
- [Neural Responding Machine for Short-Text Conversation](https://mlsanity.com/paper/1503.02364/) (2015)
- [Distilling Causal Effect of Data in Class-Incremental Learning](https://mlsanity.com/paper/2103.01737/) (2021)
- [Interventional Few-Shot Learning](https://mlsanity.com/paper/2009.13000/) (2020)
- [Causal Attention for Vision-Language Tasks](https://mlsanity.com/paper/2103.03493/) (2021)
- [A Neural Network Approach to Context-Sensitive Generation of Conversational Responses](https://mlsanity.com/paper/1506.06714/) (2015)
- [Counterfactual Zero-Shot and Open-Set Visual Recognition](https://mlsanity.com/paper/2103.00887/) (2021)
- [Graph Infomax Adversarial Learning for Treatment Effect Estimation with Networked Observational Data](https://mlsanity.com/paper/2106.02881/) (2021)
- [Sequence to Backward and Forward Sequences: A Content-Introducing Approach to Generative Short-Text Conversation](https://mlsanity.com/paper/1607.00970/) (2016)
- [Matching in Selective and Balanced Representation Space for Treatment Effects Estimation](https://mlsanity.com/paper/2009.06828/) (2020)
- [Emotion-aware Chat Machine: Automatic Emotional Response Generation for Human-like Emotional Interaction](https://mlsanity.com/paper/2106.03044/) (2021)
- [A Discrete CVAE for Response Generation on Short-Text Conversation](https://mlsanity.com/paper/1911.09845/) (2019)
- [Generating Multiple Diverse Responses for Short-Text Conversation](https://mlsanity.com/paper/1811.05696/) (2018)
- [Target Guided Emotion Aware Chat Machine](https://mlsanity.com/paper/2011.07432/) (2020)
- [Learning Infomax and Domain-Independent Representations for Causal Effect Inference with Real-World Data](https://mlsanity.com/paper/2202.10885/) (2022)
- [Generating Multiple Diverse Responses with Multi-Mapping and Posterior Mapping Selection](https://mlsanity.com/paper/1906.01781/) (2019)
- [Multi-Task Adversarial Learning for Treatment Effect Estimation in Basket Trials](https://mlsanity.com/paper/2203.05123/) (2022)
- [Hierarchical Capsule Prediction Network for Marketing Campaigns Effect](https://mlsanity.com/paper/2208.10113/) (2022)
- [Another Diversity-Promoting Objective Function for Neural Dialogue Generation](https://mlsanity.com/paper/1811.08100/) (2018)

## Built on by (most cited first, 4 of 4)

- [A Causal Explainable Guardrails for Large Language Models](https://mlsanity.com/paper/2405.04160/) (2024)
- [Personalized Topic Selection Model for Topic-Grounded Dialogue](https://mlsanity.com/paper/2406.01988/) (2024)
- [Continual Causal Effect Estimation: Challenges and Opportunities](https://mlsanity.com/paper/2301.01026/) (2023)
- [Position Debiasing Fine-Tuning for Causal Perception in Long-Term Dialogue](https://mlsanity.com/paper/2406.02002/) (2024)

## Abstract (arXiv)

Although the Conditional Variational AutoEncoder (CVAE) model can generate more diversified responses than the traditional Seq2Seq model, the responses often have low relevance with the input words or are illogical with the question. A causal analysis is carried out to study the reasons behind, and a methodology of searching for the mediators and mitigating the confounding bias in dialogues is provided. Specifically, we propose to predict the mediators to preserve relevant information and auto-regressively incorporate the mediators into generating process. Besides, a dynamic topic graph guided conditional variational autoencoder (TGG-CVAE) model is utilized to complement the semantic space and reduce the confounding bias in responses. Extensive experiments demonstrate that the proposed model is able to generate both relevant and informative responses, and outperforms the state-of-the-art in terms of automatic metrics and human evaluations.