# DisCoDisCo at the DISRPT2021 Shared Task: A System for Discourse Segmentation, Classification, and Connective Detection

arXiv: https://arxiv.org/abs/2109.09777 · HTML: https://mlsanity.com/paper/2109.09777/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 57 references that are in the map.
- First posted on arXiv 20 Sep 2021.

## Builds on (most cited first, 14 of 14)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019)
- [Enriching Word Vectors with Subword Information](https://mlsanity.com/paper/1607.04606/) (2016)
- [A Primer in BERTology: What we know about how BERT works](https://mlsanity.com/paper/2002.12327/) (2020)
- [Stanza: A Python Natural Language Processing Toolkit for Many Human Languages](https://mlsanity.com/paper/2003.07082/) (2020)
- [AllenNLP: A Deep Semantic Natural Language Processing Platform](https://mlsanity.com/paper/1803.07640/) (2018)
- [Cross-lingual RST Discourse Parsing](https://mlsanity.com/paper/1701.02946/) (2017)
- [RST Parsing from Scratch](https://mlsanity.com/paper/2105.10861/) (2021)
- [Implicit Discourse Relation Classification via Multi-Task Neural Networks](https://mlsanity.com/paper/1603.02776/) (2016)
- [Extending Implicit Discourse Relation Recognition to the PDTB-3](https://mlsanity.com/paper/2010.06294/) (2020)
- [AMALGUM -- A Free, Balanced, Multilayer English Web Corpus](https://mlsanity.com/paper/2006.10677/) (2020)
- [Beyond The Wall Street Journal: Anchoring and Comparing Discourse Signals across Genres](https://mlsanity.com/paper/1909.00516/) (2019)

## Built on by (most cited first, 15 of 15)

- [GENTLE: A Genre-Diverse Multilayer Challenge Set for English NLP and Linguistic Evaluation](https://mlsanity.com/paper/2306.01966/) (2023)
- [Incorporating Distributions of Discourse Structure for Long Document Abstractive Summarization](https://mlsanity.com/paper/2305.16784/) (2023)
- [Why Can't Discourse Parsing Generalize? A Thorough Investigation of the Impact of Data Diversity](https://mlsanity.com/paper/2302.06488/) (2023)
- [eRST: A Signaled Graph Theory of Discourse Relations and Organization](https://mlsanity.com/paper/2403.13560/) (2024)
- [Implicit Discourse Relation Classification For Nigerian Pidgin](https://mlsanity.com/paper/2406.18776/) (2024)
- [GDTB: Genre Diverse Data for English Shallow Discourse Parsing across Modalities, Text Types, and Domains](https://mlsanity.com/paper/2411.00491/) (2024)
- [Discourse Structure Extraction from Pre-Trained and Fine-Tuned Language Models in Dialogues](https://mlsanity.com/paper/2302.05895/) (2023)
- [Leveraging Small LLMs for Argument Mining in Education: Argument Component Identification, Classification, and Assessment](https://mlsanity.com/paper/2502.14389/) (2025)
- [Probing LLMs for Multilingual Discourse Generalization Through a Unified Label Set](https://mlsanity.com/paper/2503.10515/) (2025)
- [AblationBench: Evaluating Automated Planning of Ablations in Empirical AI Research](https://mlsanity.com/paper/2507.08038/) (2025)
- [DeDisCo at the DISRPT 2025 Shared Task: A System for Discourse Relation Classification](https://mlsanity.com/paper/2509.11498/) (2025)
- [On the Role of Context for Discourse Relation Classification in Scientific Writing](https://mlsanity.com/paper/2510.26354/) (2025)
- [Discourse Relations Classification and Cross-Framework Discourse Relation Classification Through the Lens of Cognitive Dimensions: An Empirical Investigation](https://mlsanity.com/paper/2311.00451/) (2023)
- [Lightweight Connective Detection Using Gradient Boosting](https://mlsanity.com/paper/2404.13793/) (2024)
- [CLaC at DISRPT 2025: Hierarchical Adapters for Cross-Framework Multi-lingual Discourse Relation Classification](https://mlsanity.com/paper/2509.16903/) (2025)

## Abstract (arXiv)

This paper describes our submission to the DISRPT2021 Shared Task on Discourse Unit Segmentation, Connective Detection, and Relation Classification. Our system, called DisCoDisCo, is a Transformer-based neural classifier which enhances contextualized word embeddings (CWEs) with hand-crafted features, relying on tokenwise sequence tagging for discourse segmentation and connective detection, and a feature-rich, encoder-less sentence pair classifier for relation classification. Our results for the first two tasks outperform SOTA scores from the previous 2019 shared task, and results on relation classification suggest strong performance on the new 2021 benchmark. Ablation tests show that including features beyond CWEs are helpful for both tasks, and a partial evaluation of multiple pre-trained Transformer-based language models indicates that models pre-trained on the Next Sentence Prediction (NSP) task are optimal for relation classification.