# Discourse Relation Embeddings: Representing the Relations between Discourse Segments in Social Media

arXiv: https://arxiv.org/abs/2105.01306 · HTML: https://mlsanity.com/paper/2105.01306/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 36 references that are in the map.
- First posted on arXiv 4 May 2021.

## Builds on (most cited first, 12 of 12)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Deep contextualized word representations](https://mlsanity.com/paper/1802.05365/) (2018)
- [Supervised Learning of Universal Sentence Representations from Natural Language Inference Data](https://mlsanity.com/paper/1705.02364/) (2017)
- [Mining Discourse Markers for Unsupervised Sentence Representation Learning](https://mlsanity.com/paper/1903.11850/) (2019)
- [Neural Discourse Structure for Text Categorization](https://mlsanity.com/paper/1702.01829/) (2017)
- [Better Document-level Sentiment Analysis from RST Discourse Parsing](https://mlsanity.com/paper/1509.01599/) (2015)
- [Deep Enhanced Representation for Implicit Discourse Relation Recognition](https://mlsanity.com/paper/1807.05154/) (2018)
- [Discourse-Based Objectives for Fast Unsupervised Sentence Representation Learning](https://mlsanity.com/paper/1705.00557/) (2017)
- [Implicit Discourse Relation Classification via Multi-Task Neural Networks](https://mlsanity.com/paper/1603.02776/) (2016)
- [A Latent Variable Recurrent Neural Network for Discourse Relation Language Models](https://mlsanity.com/paper/1603.01913/) (2016)

## Built on by (most cited first, 8 of 8)

- [A Survey of Implicit Discourse Relation Recognition](https://mlsanity.com/paper/2203.02982/) (2022)
- [Explainable Causal Analysis of Mental Health on Social Media Data](https://mlsanity.com/paper/2210.08430/) (2022)
- [NLP as a Lens for Causal Analysis and Perception Mining to Infer Mental Health on Social Media](https://mlsanity.com/paper/2301.11004/) (2023)
- [Transfer and Active Learning for Dissonance Detection: Addressing the Rare-Class Challenge](https://mlsanity.com/paper/2305.02459/) (2023)
- [Capturing Human Cognitive Styles with Language: Towards an Experimental Evaluation Paradigm](https://mlsanity.com/paper/2502.13326/) (2025)
- [Causal Categorization of Mental Health Posts using Transformers](https://mlsanity.com/paper/2301.02589/) (2023)
- [Multi-class Categorization of Reasons behind Mental Disturbance in Long Texts](https://mlsanity.com/paper/2304.04118/) (2023)
- [Embedding Mental Health Discourse for Community Recommendation](https://mlsanity.com/paper/2307.03892/) (2023)

## Abstract (arXiv)

Discourse relations are typically modeled as a discrete class that characterizes the relation between segments of text (e.g. causal explanations, expansions). However, such predefined discrete classes limits the universe of potential relationships and their nuanced differences. Analogous to contextual word embeddings, we propose representing discourse relations as points in high dimensional continuous space. However, unlike words, discourse relations often have no surface form (relations are between two segments, often with no word or phrase in that gap) which presents a challenge for existing embedding techniques. We present a novel method for automatically creating discourse relation embeddings (DiscRE), addressing the embedding challenge through a weakly supervised, multitask approach to learn diverse and nuanced relations between discourse segments in social media. Results show DiscRE can: (1) obtain the best performance on Twitter discourse relation classification task (macro F1=0.76) (2) improve the state of the art in social media causality prediction (from F1=.79 to .81), (3) perform beyond modern sentence and contextual word embeddings at traditional discourse relation classification, and (4) capture novel nuanced relations (e.g. relations semantically at the intersection of causal explanations and counterfactuals).