# Facilitating Contrastive Learning of Discourse Relational Senses by Exploiting the Hierarchy of Sense Relations

arXiv: https://arxiv.org/abs/2301.02724 · HTML: https://mlsanity.com/paper/2301.02724/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 63 references that are in the map.
- First posted on arXiv 6 Jan 2023.

## Builds on (most cited first, 29 of 29)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [Supervised Contrastive Learning](https://mlsanity.com/paper/2004.11362/) (2020)
- [Deep contextualized word representations](https://mlsanity.com/paper/1802.05365/) (2018)
- [SimCSE: Simple Contrastive Learning of Sentence Embeddings](https://mlsanity.com/paper/2104.08821/) (2021)
- [Contrastive Learning with Hard Negative Samples](https://mlsanity.com/paper/2010.04592/) (2020)
- [A Discourse-Aware Attention Model for Abstractive Summarization of Long Documents](https://mlsanity.com/paper/1804.05685/) (2018)
- [Generating Natural Language Adversarial Examples](https://mlsanity.com/paper/1804.07998/) (2018)
- [Supervised Contrastive Learning for Pre-trained Language Model Fine-tuning](https://mlsanity.com/paper/2011.01403/) (2020)
- [ConSERT: A Contrastive Framework for Self-Supervised Sentence Representation Transfer](https://mlsanity.com/paper/2105.11741/) (2021)
- [CLEAR: Contrastive Learning for Sentence Representation](https://mlsanity.com/paper/2012.15466/) (2020)
- [Self-Guided Contrastive Learning for BERT Sentence Representations](https://mlsanity.com/paper/2106.07345/) (2021)
- [CLINE: Contrastive Learning with Semantic Negative Examples for Natural Language Understanding](https://mlsanity.com/paper/2107.00440/) (2021)
- [Not All Negatives are Equal: Label-Aware Contrastive Loss for Fine-grained Text Classification](https://mlsanity.com/paper/2109.05427/) (2021)
- [On the Importance of Word and Sentence Representation Learning in Implicit Discourse Relation Classification](https://mlsanity.com/paper/2004.12617/) (2020)
- [Pairwise Supervised Contrastive Learning of Sentence Representations](https://mlsanity.com/paper/2109.05424/) (2021)
- [Exploring the Impact of Negative Samples of Contrastive Learning: A Case Study of Sentence Embedding](https://mlsanity.com/paper/2202.13093/) (2022)
- [Deep Enhanced Representation for Implicit Discourse Relation Recognition](https://mlsanity.com/paper/1807.05154/) (2018)
- [A Label Dependence-aware Sequence Generation Model for Multi-level Implicit Discourse Relation Recognition](https://mlsanity.com/paper/2112.11740/) (2021)
- [Adversarial Connective-exploiting Networks for Implicit Discourse Relation Classification](https://mlsanity.com/paper/1704.00217/) (2017)
- [Improving Implicit Discourse Relation Classification by Modeling Inter-dependencies of Discourse Units in a Paragraph](https://mlsanity.com/paper/1804.05918/) (2018)
- [Recognizing Implicit Discourse Relations via Repeated Reading: Neural Networks with Multi-Level Attention](https://mlsanity.com/paper/1609.06380/) (2016)
- [Learning to Explicitate Connectives with Seq2Seq Network for Implicit Discourse Relation Classification](https://mlsanity.com/paper/1811.01697/) (2018)
- [From Discourse to Narrative: Knowledge Projection for Event Relation Extraction](https://mlsanity.com/paper/2106.08629/) (2021)
- [Efficient Contrastive Learning via Novel Data Augmentation and Curriculum Learning](https://mlsanity.com/paper/2109.05941/) (2021)
- [Extending Implicit Discourse Relation Recognition to the PDTB-3](https://mlsanity.com/paper/2010.06294/) (2020)

## Built on by (most cited first, 22 of 22)

- [ChatGPT Evaluation on Sentence Level Relations: A Focus on Temporal, Causal, and Discourse Relations](https://mlsanity.com/paper/2304.14827/) (2023)
- [DiscoPrompt: Path Prediction Prompt Tuning for Implicit Discourse Relation Recognition](https://mlsanity.com/paper/2305.03973/) (2023)
- [Annotation-Inspired Implicit Discourse Relation Classification with Auxiliary Discourse Connective Generation](https://mlsanity.com/paper/2306.06480/) (2023)
- [Multi-Label Classification for Implicit Discourse Relation Recognition](https://mlsanity.com/paper/2406.04461/) (2024)
- [The Box is in the Pen: Evaluating Commonsense Reasoning in Neural Machine Translation](https://mlsanity.com/paper/2503.03308/) (2025)
- [A Novel Computational and Modeling Foundation for Automatic Coherence Assessment](https://mlsanity.com/paper/2310.00598/) (2023)
- [Instances and Labels: Hierarchy-aware Joint Supervised Contrastive Learning for Hierarchical Multi-Label Text Classification](https://mlsanity.com/paper/2310.05128/) (2023)
- [What Causes the Failure of Explicit to Implicit Discourse Relation Recognition?](https://mlsanity.com/paper/2404.00999/) (2024)
- [A Multi-Task and Multi-Label Classification Model for Implicit Discourse Relation Recognition](https://mlsanity.com/paper/2408.08971/) (2024)
- [BUCA: A Binary Classification Approach to Unsupervised Commonsense Question Answering](https://mlsanity.com/paper/2305.15932/) (2023)
- [Learning Label Hierarchy with Supervised Contrastive Learning](https://mlsanity.com/paper/2402.00232/) (2024)
- [Prompt-based Logical Semantics Enhancement for Implicit Discourse Relation Recognition](https://mlsanity.com/paper/2311.00367/) (2023)
- [Constrained Multi-Layer Contrastive Learning for Implicit Discourse Relationship Recognition](https://mlsanity.com/paper/2409.13716/) (2024)
- [Tgea: An error-annotated dataset and benchmark tasks for text generation from pretrained language models](https://mlsanity.com/paper/2503.04232/) (2025)
- [Fine-Grained Evaluation for Implicit Discourse Relation Recognition](https://mlsanity.com/paper/2503.05326/) (2025)
- [Evaluating Discourse Cohesion in Pre-trained Language Models](https://mlsanity.com/paper/2503.06137/) (2025)
- [Multi-Lingual Implicit Discourse Relation Recognition with Multi-Label Hierarchical Learning](https://mlsanity.com/paper/2508.20712/) (2025)
- [On the Role of Context for Discourse Relation Classification in Scientific Writing](https://mlsanity.com/paper/2510.26354/) (2025)
- [Meta-RTL: Reinforcement-Based Meta-Transfer Learning for Low-Resource Commonsense Reasoning](https://mlsanity.com/paper/2409.19075/) (2024)
- [Similarity-Based Domain Adaptation with LLMs](https://mlsanity.com/paper/2503.05281/) (2025)
- [MetaXCR: Reinforcement-Based Meta-Transfer Learning for Cross-Lingual Commonsense Reasoning](https://mlsanity.com/paper/2503.06531/) (2025)
- [Improving Implicit Discourse Relation Recognition with Natural Language Explanations from LLMs](https://mlsanity.com/paper/2602.21763/) (2026)

## Abstract (arXiv)

Implicit discourse relation recognition is a challenging task that involves identifying the sense or senses that hold between two adjacent spans of text, in the absence of an explicit connective between them. In both PDTB-2 and PDTB-3, discourse relational senses are organized into a three-level hierarchy ranging from four broad top-level senses, to more specific senses below them. Most previous work on implicit discourse relation recognition have used the sense hierarchy simply to indicate what sense labels were available. Here we do more -- incorporating the sense hierarchy into the recognition process itself and using it to select the negative examples used in contrastive learning. With no additional effort, the approach achieves state-of-the-art performance on the task.