mlsanityData to 7 Oct 2026

Evaluating Discourse Cohesion in Pre-trained Language Models

arXiv 2503.06137 ↗First posted 8 Mar 2025cs.CLcited by 1

Not on the map. Why?

Abstract (arXiv)

Large pre-trained neural models have achieved remarkable success in natural language process (NLP), inspiring a growing body of research analyzing their ability from different aspects. In this paper, we propose a test suite to evaluate the cohesive ability of pre-trained language models. The test suite contains multiple cohesion phenomena between adjacent and non-adjacent sentences. We try to compare different pre-trained language models on these phenomena and analyze the experimental results,hoping more attention can be given to discourse cohesion in the future.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 17 of its 30 references that are in the map. First posted on arXiv 8 Mar 2025.

Builds on

17 of 30 references.

References · 17

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  3. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  4. 2019How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddingscited by 770
  5. 2019BLiMP: The Benchmark of Linguistic Minimal Pairs for Englishcited by 553
  6. 2016A Corpus and Evaluation Framework for Deeper Understanding of Commonsense Storiescited by 413
  7. 2019Linguistic Knowledge and Transferability of Contextual Representationscited by 382
  8. 2019BERT for Coreference Resolution: Baselines and Analysiscited by 125
  9. 2019Multi-passage BERT: A Globally Normalized BERT Model for Open-domain Question Answeringcited by 96
  10. 2019Probing What Different NLP Tasks Teach Machines about Function Word Comprehensioncited by 44
  11. 2019DiscoFuse: A Large-Scale Dataset for Discourse-Based Sentence Fusioncited by 29
  12. 2023Facilitating Contrastive Learning of Discourse Relational Senses by Exploiting the Hierarchy of Sense Relationscited by 22
5 more
  1. 2019Evaluation Benchmarks and Learning Criteria for Discourse-Aware Sentence Representationscited by 21
  2. 2020Shallow Discourse Annotation for Chinese TED Talkscited by 13
  3. 2024Multi-Label Classification for Implicit Discourse Relation Recognitioncited by 9
  4. 2024Leveraging Hierarchical Prototypes as the Verbalizer for Implicit Discourse Relation Recognitioncited by 5
  5. 2022Quantitative Discourse Cohesion Analysis of Scientific Scholarly Texts using Multilayer Networkscited by 2

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025MetaXCR: Reinforcement-Based Meta-Transfer Learning for Cross-Lingual Commonsense Reasoningcited by 0