# Multitask Learning for Class-Imbalanced Discourse Classification

arXiv: https://arxiv.org/abs/2101.00389 · HTML: https://mlsanity.com/paper/2101.00389/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 64 references that are in the map.
- First posted on arXiv 2 Jan 2021.

## Builds on (most cited first, 30 of 30)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation](https://mlsanity.com/paper/1606.04797/) (2016)
- [Deep contextualized word representations](https://mlsanity.com/paper/1802.05365/) (2018)
- [fairseq: A Fast, Extensible Toolkit for Sequence Modeling](https://mlsanity.com/paper/1904.01038/) (2019)
- [MixMatch: A Holistic Approach to Semi-Supervised Learning](https://mlsanity.com/paper/1905.02249/) (2019)
- [Unsupervised Data Augmentation for Consistency Training](https://mlsanity.com/paper/1904.12848/) (2019)
- [A Survey on Multi-Task Learning](https://mlsanity.com/paper/1707.08114/) (2017)
- [An Overview of Multi-Task Learning in Deep Neural Networks](https://mlsanity.com/paper/1706.05098/) (2017)
- [Generalised Dice overlap as a deep learning loss function for highly unbalanced segmentations](https://mlsanity.com/paper/1707.03237/) (2017)
- [Evaluation of Text Generation: A Survey](https://mlsanity.com/paper/2006.14799/) (2020)
- [Massively Multilingual Neural Machine Translation](https://mlsanity.com/paper/1903.00089/) (2019)
- [MixText: Linguistically-Informed Interpolation of Hidden Space for Semi-Supervised Text Classification](https://mlsanity.com/paper/2004.12239/) (2020)
- [Dice Loss for Data-imbalanced NLP Tasks](https://mlsanity.com/paper/1911.02855/) (2019)
- [Long Short-Term Memory-Networks for Machine Reading](https://mlsanity.com/paper/1601.06733/) (2016)
- [Dataset Augmentation in Feature Space](https://mlsanity.com/paper/1702.05538/) (2017)
- [A Multi-Axis Annotation Scheme for Event Temporal Relations](https://mlsanity.com/paper/1804.07828/) (2018)
- [SBERT-WK: A Sentence Embedding Method by Dissecting BERT-based Word Models](https://mlsanity.com/paper/2002.06652/) (2020)
- [Multilingual Part-of-Speech Tagging with Bidirectional Long Short-Term Memory Models and Auxiliary Loss](https://mlsanity.com/paper/1604.05529/) (2016)
- [Fake News Early Detection: An Interdisciplinary Study](https://mlsanity.com/paper/1904.11679/) (2019)
- [Class-Imbalanced Semi-Supervised Learning](https://mlsanity.com/paper/2002.06815/) (2020)
- [Toward Fast and Accurate Neural Discourse Segmentation](https://mlsanity.com/paper/1808.09147/) (2018)
- [Multilingual Neural RST Discourse Parsing](https://mlsanity.com/paper/2012.01704/) (2020)
- [Implicit Discourse Relation Classification via Multi-Task Neural Networks](https://mlsanity.com/paper/1603.02776/) (2016)
- [Unsupervised Neural Single-Document Summarization of Reviews via Learning Latent Discourse Structure and its Ranking](https://mlsanity.com/paper/1906.05691/) (2019)
- [Attributed Rhetorical Structure Grammar for Domain Text Summarization](https://mlsanity.com/paper/1909.00923/) (2019)
- [Multilingual Part-of-Speech Tagging: Two Unsupervised Approaches](https://mlsanity.com/paper/1401.5695/) (2014)
- [Automatic Prediction of Discourse Connectives](https://mlsanity.com/paper/1702.00992/) (2017)
- [A Multilingual Information Extraction Pipeline for Investigative Journalism](https://mlsanity.com/paper/1809.00221/) (2018)

## Built on by (most cited first, 2 of 2)

- [NewsEdits: A Dataset of Revision Histories for News Articles (Technical Report: Data Processing)](https://mlsanity.com/paper/2104.09647/) (2021)
- [StateCensusLaws.org: A Web Application for Consuming and Annotating Legal Discourse Learning](https://mlsanity.com/paper/2104.10263/) (2021)

## Abstract (arXiv)

Small class-imbalanced datasets, common in many high-level semantic tasks like discourse analysis, present a particular challenge to current deep-learning architectures. In this work, we perform an extensive analysis on sentence-level classification approaches for the News Discourse dataset, one of the largest high-level semantic discourse datasets recently published. We show that a multitask approach can improve 7% Micro F1-score upon current state-of-the-art benchmarks, due in part to label corrections across tasks, which improve performance for underrepresented classes. We also offer a comparative review of additional techniques proposed to address resource-poor problems in NLP, and show that none of these approaches can improve classification accuracy in such a setting.