# Hierarchical Recurrent Attention Network for Response Generation

arXiv: https://arxiv.org/abs/1701.07149 · HTML: https://mlsanity.com/paper/1701.07149/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Jan 2017.

## Built on by (most cited first, 30 of 34)

- [Recent Advances in Deep Learning Based Dialogue Systems: A Systematic Survey](https://mlsanity.com/paper/2105.04387/) (2021)
- [A General Survey on Attention Mechanisms in Deep Learning](https://mlsanity.com/paper/2203.14263/) (2022)
- [Attention, please! A survey of Neural Attention Models in Deep Learning](https://mlsanity.com/paper/2103.16775/) (2021)
- [Hierarchical Learning for Generation with Long Source Sequences](https://mlsanity.com/paper/2104.07545/) (2021)
- [Multimodal Dialogue Response Generation](https://mlsanity.com/paper/2110.08515/) (2021)
- [FADO: Feedback-Aware Double COntrolling Network for Emotional Support Conversation](https://mlsanity.com/paper/2211.00250/) (2022)
- [GTM: A Generative Triple-Wise Model for Conversational Question Generation](https://mlsanity.com/paper/2106.03635/) (2021)
- [Learning to Select Context in a Hierarchical and Global Perspective for Open-domain Dialogue Generation](https://mlsanity.com/paper/2102.09282/) (2021)
- [Dialogue History Matters! Personalized Response Selectionin Multi-turn Retrieval-based Chatbots](https://mlsanity.com/paper/2103.09534/) (2021)
- [BERT-CoQAC: BERT-based Conversational Question Answering in Context](https://mlsanity.com/paper/2104.11394/) (2021)
- [PopALM: Popularity-Aligned Language Models for Social Media Trendy Response Prediction](https://mlsanity.com/paper/2402.18950/) (2024)
- [Empathetic Dialog Generation with Fine-Grained Intents](https://mlsanity.com/paper/2105.06829/) (2021)
- [Retrieval-Augmented Transformer-XL for Close-Domain Dialog Generation](https://mlsanity.com/paper/2105.09235/) (2021)
- [Progressive Open-Domain Response Generation with Multiple Controllable Attributes](https://mlsanity.com/paper/2106.14614/) (2021)
- [PVGRU: Generating Diverse and Relevant Dialogue Responses via Pseudo-Variational Mechanism](https://mlsanity.com/paper/2212.09086/) (2022)
- [Robust Task-Oriented Dialogue Generation with Contrastive Pre-training and Adversarial Filtering](https://mlsanity.com/paper/2205.10363/) (2022)
- [CauESC: A Causal Aware Model for Emotional Support Conversation](https://mlsanity.com/paper/2401.17755/) (2024)
- [Transformer Based Bengali Chatbot Using General Knowledge Dataset](https://mlsanity.com/paper/2111.03937/) (2021)
- [WeaSuL: Weakly Supervised Dialogue Policy Learning: Reward Estimation for Multi-turn Dialogue](https://mlsanity.com/paper/2108.01487/) (2021)
- [A Speaker-aware Parallel Hierarchical Attentive Encoder-Decoder Model for Multi-turn Dialogue Generation](https://mlsanity.com/paper/2110.06823/) (2021)
- [What Did You Say? Task-Oriented Dialog Datasets Are Not Conversational!?](https://mlsanity.com/paper/2203.03431/) (2022)
- [Learning to Express in Knowledge-Grounded Conversation](https://mlsanity.com/paper/2204.05805/) (2022)
- [RT-KGD: Relation Transition Aware Knowledge-Grounded Dialogue Generation](https://mlsanity.com/paper/2207.08212/) (2022)
- [DialoGen: Generalized Long-Range Context Representation for Dialogue Systems](https://mlsanity.com/paper/2210.06282/) (2022)
- [Know Deeper: Knowledge-Conversation Cyclic Utilization Mechanism for Open-domain Dialogue Generation](https://mlsanity.com/paper/2107.07771/) (2021)
- [Towards Explanation for Unsupervised Graph-Level Representation Learning](https://mlsanity.com/paper/2205.09934/) (2022)
- [Learning Locality and Isotropy in Dialogue Modeling](https://mlsanity.com/paper/2205.14583/) (2022)
- [Query-Response Interactions by Multi-tasks in Semantic Search for Chatbot Candidate Retrieval](https://mlsanity.com/paper/2208.11018/) (2022)
- [Generating Persuasive Responses to Customer Reviews with Multi-Source Prior Knowledge in E-commerce](https://mlsanity.com/paper/2209.09497/) (2022)
- [X-ReCoSa: Multi-Scale Context Aggregation For Multi-Turn Dialogue Generation](https://mlsanity.com/paper/2303.07833/) (2023)

## Abstract (arXiv)

We study multi-turn response generation in chatbots where a response is generated according to a conversation context. Existing work has modeled the hierarchy of the context, but does not pay enough attention to the fact that words and utterances in the context are differentially important. As a result, they may lose important information in context and generate irrelevant responses. We propose a hierarchical recurrent attention network (HRAN) to model both aspects in a unified framework. In HRAN, a hierarchical attention mechanism attends to important parts within and among utterances with word level attention and utterance level attention respectively. With the word level attention, hidden vectors of a word level encoder are synthesized as utterance vectors and fed to an utterance level encoder to construct hidden representations of the context. The hidden vectors of the context are then processed by the utterance level attention and formed as context vectors for decoding the response. Empirical studies on both automatic evaluation and human judgment show that HRAN can significantly outperform state-of-the-art models for multi-turn response generation.