# Multiresolution Recurrent Neural Networks: An Application to Dialogue Response Generation

arXiv: https://arxiv.org/abs/1606.00776 · HTML: https://mlsanity.com/paper/1606.00776/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Jun 2016.

## Built on by (most cited first, 16 of 16)

- [On Decoding Strategies for Neural Text Generators](https://mlsanity.com/paper/2203.15721/) (2022)
- [CHAI: A CHatbot AI for Task-Oriented Dialogue with Offline Reinforcement Learning](https://mlsanity.com/paper/2204.08426/) (2022)
- [Determinantal Beam Search](https://mlsanity.com/paper/2106.07400/) (2021)
- [Learning to Ask Like a Physician](https://mlsanity.com/paper/2206.02696/) (2022)
- [Dialogue History Matters! Personalized Response Selectionin Multi-turn Retrieval-based Chatbots](https://mlsanity.com/paper/2103.09534/) (2021)
- [Conditional Poisson Stochastic Beam Search](https://mlsanity.com/paper/2109.11034/) (2021)
- [Retrieval-Augmented Transformer-XL for Close-Domain Dialog Generation](https://mlsanity.com/paper/2105.09235/) (2021)
- [Multi-View Feature Representation for Dialogue Generation with Bidirectional Distillation](https://mlsanity.com/paper/2102.10780/) (2021)
- [Automatic Controllable Product Copywriting for E-Commerce](https://mlsanity.com/paper/2206.10103/) (2022)
- [Understanding and Improving the Exemplar-based Generation for Open-domain Conversation](https://mlsanity.com/paper/2112.06723/) (2021)
- [Stop Filtering: Multi-View Attribute-Enhanced Dialogue Learning](https://mlsanity.com/paper/2205.11206/) (2022)
- [From Intents to Conversations: Generating Intent-Driven Dialogues with Contrastive Learning for Multi-Turn Classification](https://mlsanity.com/paper/2411.14252/) (2024)
- [Heterogeneous-Branch Collaborative Learning for Dialogue Generation](https://mlsanity.com/paper/2303.11621/) (2023)
- [Local and Global Contexts for Conversation](https://mlsanity.com/paper/2401.17588/) (2024)
- [Synthetic Test Data Generation Using Recurrent Neural Networks: A Position Paper](https://mlsanity.com/paper/2407.05410/) (2024)
- [A Static and Dynamic Attention Framework for Multi Turn Dialogue Generation](https://mlsanity.com/paper/2410.20766/) (2024)

## Abstract (arXiv)

We introduce the multiresolution recurrent neural network, which extends the sequence-to-sequence framework to model natural language generation as two parallel discrete stochastic processes: a sequence of high-level coarse tokens, and a sequence of natural language tokens. There are many ways to estimate or learn the high-level coarse tokens, but we argue that a simple extraction procedure is sufficient to capture a wealth of high-level discourse semantics. Such procedure allows training the multiresolution recurrent neural network by maximizing the exact joint log-likelihood over both sequences. In contrast to the standard log- likelihood objective w.r.t. natural language tokens (word perplexity), optimizing the joint log-likelihood biases the model towards modeling high-level abstractions. We apply the proposed model to the task of dialogue response generation in two challenging domains: the Ubuntu technical support domain, and Twitter conversations. On Ubuntu, the model outperforms competing approaches by a substantial margin, achieving state-of-the-art results according to both automatic evaluation metrics and a human evaluation study. On Twitter, the model appears to generate more relevant and on-topic responses according to automatic evaluation metrics. Finally, our experiments demonstrate that the proposed model is more adept at overcoming the sparsity of natural language and is better able to capture long-term structure.