# Generating More Interesting Responses in Neural Conversation Models with Distributional Constraints

arXiv: https://arxiv.org/abs/1809.01215 · HTML: https://mlsanity.com/paper/1809.01215/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Sep 2018.

## Built on by (most cited first, 30 of 33)

- [Offline RL for Natural Language Generation with Implicit Language Q Learning](https://mlsanity.com/paper/2206.11871/) (2022)
- [Grounded Decoding: Guiding Text Generation with Grounded Models for Embodied Agents](https://mlsanity.com/paper/2303.00855/) (2023)
- [Defending Against Backdoor Attacks in Natural Language Generation](https://mlsanity.com/paper/2106.01810/) (2021)
- [Personalized Showcases: Generating Multi-Modal Explanations for Recommendations](https://mlsanity.com/paper/2207.00422/) (2022)
- [Interpreting Deep Learning Models in Natural Language Processing: A Review](https://mlsanity.com/paper/2110.10470/) (2021)
- [Why is constrained neural language generation particularly challenging?](https://mlsanity.com/paper/2206.05395/) (2022)
- [Generating Relevant and Coherent Dialogue Responses using Self-separated Conditional Variational AutoEncoders](https://mlsanity.com/paper/2106.03410/) (2021)
- [OpenViDial 2.0: A Larger-Scale, Open-Domain Dialogue Generation Dataset with Visual Contexts](https://mlsanity.com/paper/2109.12761/) (2021)
- [SHARE: a System for Hierarchical Assistive Recipe Editing](https://mlsanity.com/paper/2105.08185/) (2021)
- [Learning from Perturbations: Diverse and Informative Dialogue Generation with Inverse Adversarial Training](https://mlsanity.com/paper/2105.15171/) (2021)
- [Knowledge Infused Decoding](https://mlsanity.com/paper/2204.03084/) (2022)
- [Attention Is Indeed All You Need: Semantically Attention-Guided Decoding for Data-to-Text NLG](https://mlsanity.com/paper/2109.07043/) (2021)
- [Modeling Text-visual Mutual Dependency for Multi-modal Dialog Generation](https://mlsanity.com/paper/2105.14445/) (2021)
- [What Makes Data-to-Text Generation Hard for Pretrained Language Models?](https://mlsanity.com/paper/2205.11505/) (2022)
- [Towards Standard Criteria for human evaluation of Chatbots: A Survey](https://mlsanity.com/paper/2105.11197/) (2021)
- [Zero-Shot Controlled Generation with Encoder-Decoder Transformers](https://mlsanity.com/paper/2106.06411/) (2021)
- [SideControl: Controlled Open-domain Dialogue Generation via Additive Side Networks](https://mlsanity.com/paper/2109.01958/) (2021)
- [Controllable Dialogue Generation with Disentangled Multi-grained Style Specification and Attribute Consistency Reward](https://mlsanity.com/paper/2109.06717/) (2021)
- [Long-term Control for Dialogue Generation: Methods and Evaluation](https://mlsanity.com/paper/2205.07352/) (2022)
- [CrowdCounter: A benchmark type-specific multi-target counterspeech dataset](https://mlsanity.com/paper/2410.01400/) (2024)
- [Improving Response Quality with Backward Reasoning in Open-domain Dialogue Systems](https://mlsanity.com/paper/2105.00079/) (2021)
- [A Data-Driven Guided Decoding Mechanism for Diagnostic Captioning](https://mlsanity.com/paper/2406.14164/) (2024)
- [High-Quality Diversification for Task-Oriented Dialogue Systems](https://mlsanity.com/paper/2106.00891/) (2021)
- [WeaSuL: Weakly Supervised Dialogue Policy Learning: Reward Estimation for Multi-turn Dialogue](https://mlsanity.com/paper/2108.01487/) (2021)
- [Assistive Recipe Editing through Critiquing](https://mlsanity.com/paper/2205.02454/) (2022)
- [Stop Filtering: Multi-View Attribute-Enhanced Dialogue Learning](https://mlsanity.com/paper/2205.11206/) (2022)
- [Emotion Eliciting Machine: Emotion Eliciting Conversation Generation based on Dual Generator](https://mlsanity.com/paper/2105.08251/) (2021)
- [Neural Conversation Models and How to Rein Them in: A Survey of Failures and Fixes](https://mlsanity.com/paper/2308.06095/) (2023)
- [Contrastive Perplexity for Controlled Generation: An Application in Detoxifying Large Language Models](https://mlsanity.com/paper/2401.08491/) (2024)
- [Topical Language Generation using Transformers](https://mlsanity.com/paper/2103.06434/) (2021)

## Abstract (arXiv)

Neural conversation models tend to generate safe, generic responses for most inputs. This is due to the limitations of likelihood-based decoding objectives in generation tasks with diverse outputs, such as conversation. To address this challenge, we propose a simple yet effective approach for incorporating side information in the form of distributional constraints over the generated responses. We propose two constraints that help generate more content rich responses that are based on a model of syntax and topics (Griffiths et al., 2005) and semantic similarity (Arora et al., 2016). We evaluate our approach against a variety of competitive baselines, using both automatic metrics and human judgments, showing that our proposed approach generates responses that are much less generic without sacrificing plausibility. A working demo of our code can be found at https://github.com/abaheti95/DC-NeuralConversation.