# Improving Topic Segmentation by Injecting Discourse Dependencies

arXiv: https://arxiv.org/abs/2209.08626 · HTML: https://mlsanity.com/paper/2209.08626/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 56 references that are in the map.
- First posted on arXiv 18 Sep 2022.

## Builds on (most cited first, 24 of 24)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)](https://mlsanity.com/paper/1511.07289/) (2015)
- [Discourse-Aware Neural Extractive Text Summarization](https://mlsanity.com/paper/1910.14142/) (2019)
- [Text Segmentation as a Supervised Learning Task](https://mlsanity.com/paper/1803.09337/) (2018)
- [GRADE: Automatic Graph-Enhanced Coherence Metric for Evaluating Open-Domain Dialogue Systems](https://mlsanity.com/paper/2010.03994/) (2020)
- [Extractive Summarization of Long Documents by Combining Global and Local Context](https://mlsanity.com/paper/1909.08089/) (2019)
- [SECTOR: A Neural Model for Coherent Topic Segmentation and Classification](https://mlsanity.com/paper/1902.04793/) (2019)
- [Topic-Aware Multi-turn Dialogue Modeling](https://mlsanity.com/paper/2009.12539/) (2020)
- [Two-Level Transformer and Auxiliary Coherence Modeling for Improved Text Segmentation](https://mlsanity.com/paper/2001.00891/) (2020)
- [Unsupervised Topic Segmentation of Meetings with BERT Embeddings](https://mlsanity.com/paper/2106.12978/) (2021)
- [Improving Unsupervised Dialogue Topic Segmentation with Utterance-Pair Coherence Scoring](https://mlsanity.com/paper/2106.06719/) (2021)
- [Dependency-Guided LSTM-CRF for Named Entity Recognition](https://mlsanity.com/paper/1909.10148/) (2019)
- [Learning Structured Text Representations](https://mlsanity.com/paper/1705.09207/) (2017)
- [SciDTB: Discourse Dependency TreeBank for Scientific Abstracts](https://mlsanity.com/paper/1806.03653/) (2018)
- [Transformer over Pre-trained Transformer for Neural Text Segmentation with Enhanced Topic Coherence](https://mlsanity.com/paper/2110.07160/) (2021)
- [Improving Context Modeling in Neural Topic Segmentation](https://mlsanity.com/paper/2010.03138/) (2020)
- [Modeling Topical Relevance for Multi-Turn Dialogue Generation](https://mlsanity.com/paper/2009.12735/) (2020)
- [Dialogue Session Segmentation by Embedding-Enhanced TextTiling](https://mlsanity.com/paper/1610.03955/) (2016)
- [Sentence Similarity Based on Contexts](https://mlsanity.com/paper/2105.07623/) (2021)
- [Predicting Discourse Structure using Distant Supervision from Sentiment](https://mlsanity.com/paper/1910.14176/) (2019)
- [Unleashing the Power of Neural Discourse Parsers -- A Context and Structure Aware Approach Using Large Scale Pretraining](https://mlsanity.com/paper/2011.03203/) (2020)
- [Predicting Above-Sentence Discourse Structure using Distant Supervision from Topic Segmentation](https://mlsanity.com/paper/2112.06196/) (2021)
- [Improve Discourse Dependency Parsing with Contextualized Representations](https://mlsanity.com/paper/2205.02090/) (2022)

## Built on by (most cited first, 5 of 5)

- [eRST: A Signaled Graph Theory of Discourse Relations and Organization](https://mlsanity.com/paper/2403.13560/) (2024)
- [Advancing Topic Segmentation and Outline Generation in Chinese Texts: The Paragraph-level Topic Representation, Corpus, and Benchmark](https://mlsanity.com/paper/2305.14790/) (2023)
- [Unsupervised Mutual Learning of Discourse Parsing and Topic Segmentation in Dialogue](https://mlsanity.com/paper/2405.19799/) (2024)
- [Leveraging Collection-Wide Similarities for Unsupervised Document Structure Extraction](https://mlsanity.com/paper/2402.13906/) (2024)
- [Multi-Modal Video Topic Segmentation with Dual-Contrastive Domain Adaptation](https://mlsanity.com/paper/2312.00220/) (2023)

## Abstract (arXiv)

Recent neural supervised topic segmentation models achieve distinguished superior effectiveness over unsupervised methods, with the availability of large-scale training corpora sampled from Wikipedia. These models may, however, suffer from limited robustness and transferability caused by exploiting simple linguistic cues for prediction, but overlooking more important inter-sentential topical consistency. To address this issue, we present a discourse-aware neural topic segmentation model with the injection of above-sentence discourse dependency structures to encourage the model make topic boundary prediction based more on the topical consistency between sentences. Our empirical study on English evaluation datasets shows that injecting above-sentence discourse structures to a neural topic segmenter with our proposed strategy can substantially improve its performances on intra-domain and out-of-domain data, with little increase of model's complexity.