# Probing LLMs for Multilingual Discourse Generalization Through a Unified Label Set

arXiv: https://arxiv.org/abs/2503.10515 · HTML: https://mlsanity.com/paper/2503.10515/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 83 references that are in the map.
- First posted on arXiv 13 Mar 2025.

## Builds on (most cited first, 30 of 31)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [BLOOM: A 176B-Parameter Open-Access Multilingual Language Model](https://mlsanity.com/paper/2211.05100/) (2022)
- [Understanding intermediate layers using linear classifier probes](https://mlsanity.com/paper/1610.01644/) (2016)
- [A Discourse-Aware Attention Model for Abstractive Summarization of Long Documents](https://mlsanity.com/paper/1804.05685/) (2018)
- [What you can cram into a single vector: Probing sentence embeddings for linguistic properties](https://mlsanity.com/paper/1805.01070/) (2018)
- [What do you learn from context? Probing for sentence structure in contextualized word representations](https://mlsanity.com/paper/1905.06316/) (2019)
- [Layer by Layer: Uncovering Hidden Representations in Language Models](https://mlsanity.com/paper/2502.02013/) (2025)
- [Do Llamas Work in English? On the Latent Language of Multilingual Transformers](https://mlsanity.com/paper/2402.10588/) (2024)
- [Discourse-Aware Neural Extractive Text Summarization](https://mlsanity.com/paper/1910.14142/) (2019)
- [State-of-the-art generalisation research in NLP: A taxonomy and review](https://mlsanity.com/paper/2210.03050/) (2022)
- [How Large Language Models Encode Context Knowledge? A Layer-Wise Probing Study](https://mlsanity.com/paper/2402.16061/) (2024)
- [Impact of Tokenization on Language Models: An Analysis for Turkish](https://mlsanity.com/paper/2204.08832/) (2022)
- [Molweni: A Challenge Multiparty Dialogues-based Machine Reading Comprehension Dataset with Discourse Structure](https://mlsanity.com/paper/2004.05080/) (2020)
- [Large Language Models Share Representations of Latent Grammatical Concepts Across Typologically Diverse Languages](https://mlsanity.com/paper/2501.06346/) (2025)
- [Discourse Probing of Pretrained Language Models](https://mlsanity.com/paper/2104.05882/) (2021)
- [The Better Your Syntax, the Better Your Semantics? Probing Pretrained Language Models for the English Comparative Correlative](https://mlsanity.com/paper/2210.13181/) (2022)
- [Learning-Based Single-Document Summarization with Compression and Anaphoricity Constraints](https://mlsanity.com/paper/1603.08887/) (2016)
- [Language Models Use Monotonicity to Assess NPI Licensing](https://mlsanity.com/paper/2105.13818/) (2021)
- [DMRST: A Joint Framework for Document-Level Multilingual RST Discourse Segmentation and Parsing](https://mlsanity.com/paper/2110.04518/) (2021)
- [Cross-lingual RST Discourse Parsing](https://mlsanity.com/paper/1701.02946/) (2017)
- [DisCoDisCo at the DISRPT2021 Shared Task: A System for Discourse Segmentation, Classification, and Connective Detection](https://mlsanity.com/paper/2109.09777/) (2021)
- [Why Can't Discourse Parsing Generalize? A Thorough Investigation of the Impact of Data Diversity](https://mlsanity.com/paper/2302.06488/) (2023)
- [Sequence Tagging with Contextual and Non-Contextual Subword Representations: A Multilingual Evaluation](https://mlsanity.com/paper/1906.01569/) (2019)
- [eRST: A Signaled Graph Theory of Discourse Relations and Organization](https://mlsanity.com/paper/2403.13560/) (2024)
- [Concept Space Alignment in Multilingual LLMs](https://mlsanity.com/paper/2410.01079/) (2024)
- [Probing Multilingual Language Models for Discourse](https://mlsanity.com/paper/2106.04832/) (2021)
- [Generating EDU Extracts for Plan-Guided Summary Re-Ranking](https://mlsanity.com/paper/2305.17779/) (2023)
- [Predicting Discourse Structure using Distant Supervision from Sentiment](https://mlsanity.com/paper/1910.14176/) (2019)

## Built on by (most cited first, 4 of 4)

- [Discursive Circuits: How Do Language Models Understand Discourse Relations?](https://mlsanity.com/paper/2510.11210/) (2025)
- [Grounded Misunderstandings in Asymmetric Dialogue: A Perspectivist Annotation Scheme for MapTask](https://mlsanity.com/paper/2511.03718/) (2025)
- [CLaC at DISRPT 2025: Hierarchical Adapters for Cross-Framework Multi-lingual Discourse Relation Classification](https://mlsanity.com/paper/2509.16903/) (2025)
- [DiscoSign: Discourse-Aware Text to Sign Language Gloss Translation](https://mlsanity.com/paper/2609.02796/) (2026)

## Abstract (arXiv)

Discourse understanding is essential for many NLP tasks, yet most existing work remains constrained by framework-dependent discourse representations. This work investigates whether large language models (LLMs) capture discourse knowledge that generalizes across languages and frameworks. We address this question along two dimensions: (1) developing a unified discourse relation label set to facilitate cross-lingual and cross-framework discourse analysis, and (2) probing LLMs to assess whether they encode generalizable discourse abstractions. Using multilingual discourse relation classification as a testbed, we examine a comprehensive set of 23 LLMs of varying sizes and multilingual capabilities. Our results show that LLMs, especially those with multilingual training corpora, can generalize discourse information across languages and frameworks. Further layer-wise analyses reveal that language generalization at the discourse level is most salient in the intermediate layers. Lastly, our error analysis provides an account of challenging relation classes.