# Learning-by-Narrating: Narrative Pre-Training for Zero-Shot Dialogue Comprehension

arXiv: https://arxiv.org/abs/2203.10249 · HTML: https://mlsanity.com/paper/2203.10249/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 28 references that are in the map.
- First posted on arXiv 19 Mar 2022.

## Builds on (most cited first, 18 of 18)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Get To The Point: Summarization with Pointer-Generator Networks](https://mlsanity.com/paper/1704.04368/) (2017)
- [SAMSum Corpus: A Human-annotated Dialogue Dataset for Abstractive Summarization](https://mlsanity.com/paper/1911.12237/) (2019)
- [Newsroom: A Dataset of 1.3 Million Summaries with Diverse Extractive Strategies](https://mlsanity.com/paper/1804.11283/) (2018)
- [MovieNet: A Holistic Dataset for Movie Understanding](https://mlsanity.com/paper/2007.10937/) (2020)
- [TOD-BERT: Pre-trained Natural Language Understanding for Task-Oriented Dialogue](https://mlsanity.com/paper/2004.06871/) (2020)
- [DREAM: A Challenge Dataset and Models for Dialogue-Based Reading Comprehension](https://mlsanity.com/paper/1902.00164/) (2019)
- [MuTual: A Dataset for Multi-Turn Dialogue Reasoning](https://mlsanity.com/paper/2004.04494/) (2020)
- [DocNLI: A Large-scale Dataset for Document-level Natural Language Inference](https://mlsanity.com/paper/2106.09449/) (2021)
- [What is More Likely to Happen Next? Video-and-Language Future Event Prediction](https://mlsanity.com/paper/2010.07999/) (2020)
- [Can NLI Models Verify QA Systems' Predictions?](https://mlsanity.com/paper/2104.08731/) (2021)
- [Hierarchical Pre-training for Sequence Labelling in Spoken Dialog](https://mlsanity.com/paper/2009.11152/) (2020)
- [Conversational Neuro-Symbolic Commonsense Reasoning](https://mlsanity.com/paper/2006.10022/) (2020)
- [MMM: Multi-stage Multi-task Learning for Multi-choice Reading Comprehension](https://mlsanity.com/paper/1910.00458/) (2019)
- [Structural Pre-training for Dialogue Comprehension](https://mlsanity.com/paper/2105.10956/) (2021)
- [A Graph Reasoning Network for Multi-turn Response Selection via Customized Pre-training](https://mlsanity.com/paper/2012.11099/) (2020)
- [Different Strokes for Different Folks: Investigating Appropriate Further Pre-training Approaches for Diverse Dialogue Tasks](https://mlsanity.com/paper/2109.06524/) (2021)

## Built on by (most cited first, 3 of 3)

- [Dialogue Chain-of-Thought Distillation for Commonsense-aware Conversational Agents](https://mlsanity.com/paper/2310.09343/) (2023)
- [NarraSum: A Large-Scale Dataset for Abstractive Narrative Summarization](https://mlsanity.com/paper/2212.01476/) (2022)
- [MEGA: Multimodal Alignment Aggregation and Distillation For Cinematic Video Segmentation](https://mlsanity.com/paper/2308.11185/) (2023)

## Abstract (arXiv)

Comprehending a dialogue requires a model to capture diverse kinds of key information in the utterances, which are either scattered around or implicitly implied in different turns of conversations. Therefore, dialogue comprehension requires diverse capabilities such as paraphrasing, summarizing, and commonsense reasoning. Towards the objective of pre-training a zero-shot dialogue comprehension model, we develop a novel narrative-guided pre-training strategy that learns by narrating the key information from a dialogue input. However, the dialogue-narrative parallel corpus for such a pre-training strategy is currently unavailable. For this reason, we first construct a dialogue-narrative parallel corpus by automatically aligning movie subtitles and their synopses. We then pre-train a BART model on the data and evaluate its performance on four dialogue-based tasks that require comprehension. Experimental results show that our model not only achieves superior zero-shot performance but also exhibits stronger fine-grained dialogue comprehension capabilities. The data and code are available at https://github.com/zhaochaocs/Diana