# Improving Context Modeling in Neural Topic Segmentation

arXiv: https://arxiv.org/abs/2010.03138 · HTML: https://mlsanity.com/paper/2010.03138/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Oct 2020.

## Built on by (most cited first, 21 of 21)

- [Improving Unsupervised Dialogue Topic Segmentation with Utterance-Pair Coherence Scoring](https://mlsanity.com/paper/2106.06719/) (2021)
- [Toward Unifying Text Segmentation and Long Document Summarization](https://mlsanity.com/paper/2210.16422/) (2022)
- [Uncovering the Potential of ChatGPT for Discourse Analysis in Dialogue: An Empirical Study](https://mlsanity.com/paper/2305.08391/) (2023)
- [Transformer over Pre-trained Transformer for Neural Text Segmentation with Enhanced Topic Coherence](https://mlsanity.com/paper/2110.07160/) (2021)
- [Detecting AI-Generated Sentences in Human-AI Collaborative Hybrid Texts: Challenges, Strategies, and Insights](https://mlsanity.com/paper/2403.03506/) (2024)
- [Improving Long Document Topic Segmentation Models With Enhanced Coherence Modeling](https://mlsanity.com/paper/2310.11772/) (2023)
- [Demoting the Lead Bias in News Summarization via Alternating Adversarial Learning](https://mlsanity.com/paper/2105.14241/) (2021)
- [Human Guided Exploitation of Interpretable Attention Patterns in Summarization and Topic Segmentation](https://mlsanity.com/paper/2112.05364/) (2021)
- [Structured Summarization: Unified Text Segmentation and Segment Labeling as a Generation Task](https://mlsanity.com/paper/2209.13759/) (2022)
- [Recent Trends in Linear Text Segmentation: a Survey](https://mlsanity.com/paper/2411.16613/) (2024)
- [Predicting Above-Sentence Discourse Structure using Distant Supervision from Topic Segmentation](https://mlsanity.com/paper/2112.06196/) (2021)
- [Improving Topic Segmentation by Injecting Discourse Dependencies](https://mlsanity.com/paper/2209.08626/) (2022)
- [Topic Segmentation Model Focusing on Local Context](https://mlsanity.com/paper/2301.01935/) (2023)
- [Advancing Topic Segmentation and Outline Generation in Chinese Texts: The Paragraph-level Topic Representation, Corpus, and Benchmark](https://mlsanity.com/paper/2305.14790/) (2023)
- [Multimodal Fusion and Coherence Modeling for Video Topic Segmentation](https://mlsanity.com/paper/2408.00365/) (2024)
- [Topic-driven Distant Supervision Framework for Macro-level Discourse Parsing](https://mlsanity.com/paper/2305.13755/) (2023)
- [Grounded Complex Task Segmentation for Conversational Assistants](https://mlsanity.com/paper/2309.11271/) (2023)
- [Tracing Influence at Scale: A Contrastive Learning Approach to Linking Public Comments and Regulator Responses](https://mlsanity.com/paper/2311.14871/) (2023)
- [Multi-Modal Video Topic Segmentation with Dual-Contrastive Domain Adaptation](https://mlsanity.com/paper/2312.00220/) (2023)
- [Codebook-Injected Dialogue Segmentation for Multi-Utterance Constructs Annotation: LLM-Assisted and Gold-Label-Free Evaluation](https://mlsanity.com/paper/2601.12061/) (2026)
- [Toward General Semantic Chunking: A Discriminative Framework for Ultra-Long Documents](https://mlsanity.com/paper/2602.23370/) (2025)

## Abstract (arXiv)

Topic segmentation is critical in key NLP tasks and recent works favor highly effective neural supervised approaches. However, current neural solutions are arguably limited in how they model context. In this paper, we enhance a segmenter based on a hierarchical attention BiLSTM network to better model context, by adding a coherence-related auxiliary task and restricted self-attention. Our optimized segmenter outperforms SOTA approaches when trained and tested on three datasets. We also the robustness of our proposed model in domain transfer setting by training a model on a large-scale dataset and testing it on four challenging real-world benchmarks. Furthermore, we apply our proposed strategy to two other languages (German and Chinese), and show its effectiveness in multilingual scenarios.