# A Graph Reasoning Network for Multi-turn Response Selection via Customized Pre-training

arXiv: https://arxiv.org/abs/2012.11099 · HTML: https://mlsanity.com/paper/2012.11099/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Dec 2020.

## Built on by (most cited first, 9 of 9)

- [Advances in Multi-turn Dialogue Comprehension: A Survey](https://mlsanity.com/paper/2103.03125/) (2021)
- [Back to the Future: Bidirectional Information Decoupling Network for Multi-turn Dialogue Modeling](https://mlsanity.com/paper/2204.08152/) (2022)
- [Learning-by-Narrating: Narrative Pre-Training for Zero-Shot Dialogue Comprehension](https://mlsanity.com/paper/2203.10249/) (2022)
- [IRRGN: An Implicit Relational Reasoning Graph Network for Multi-turn Response Selection](https://mlsanity.com/paper/2212.00482/) (2022)
- [DialogConv: A Lightweight Fully Convolutional Network for Multi-view Response Selection](https://mlsanity.com/paper/2210.13845/) (2022)
- [Intent-calibrated Self-training for Answer Selection in Open-domain Dialogues](https://mlsanity.com/paper/2307.06703/) (2023)
- [Advances in Multi-turn Dialogue Comprehension: A Survey](https://mlsanity.com/paper/2110.04984/) (2021)
- [Re3Dial: Retrieve, Reorganize and Rescale Dialogue Corpus for Long-Turn Open-Domain Dialogue Pre-training](https://mlsanity.com/paper/2305.02606/) (2023)
- [Multi-turn Response Selection with Commonsense-enhanced Language Models](https://mlsanity.com/paper/2407.18479/) (2024)

## Abstract (arXiv)

We investigate response selection for multi-turn conversation in retrieval-based chatbots. Existing studies pay more attention to the matching between utterances and responses by calculating the matching score based on learned features, leading to insufficient model reasoning ability. In this paper, we propose a graph-reasoning network (GRN) to address the problem. GRN first conducts pre-training based on ALBERT using next utterance prediction and utterance order prediction tasks specifically devised for response selection. These two customized pre-training tasks can endow our model with the ability of capturing semantical and chronological dependency between utterances. We then fine-tune the model on an integrated network with sequence reasoning and graph reasoning structures. The sequence reasoning module conducts inference based on the highly summarized context vector of utterance-response pairs from the global perspective. The graph reasoning module conducts the reasoning on the utterance-level graph neural network from the local perspective. Experiments on two conversational reasoning datasets show that our model can dramatically outperform the strong baseline methods and can achieve performance which is close to human-level.