mlsanityData to 14 Oct 2026

GENTLE: A Genre-Diverse Multilayer Challenge Set for English NLP and Linguistic Evaluation

arXiv 2306.01966 ↗First posted 3 Jun 2023cs.CLcited by 19

Not on the map. Why?

Abstract (arXiv)

We present GENTLE, a new mixed-genre English challenge corpus totaling 17K tokens and consisting of 8 unusual text types for out-of domain evaluation: dictionary entries, esports commentaries, legal documents, medical notes, poetry, mathematical proofs, syllabuses, and threat letters. GENTLE is manually annotated for a variety of popular NLP tasks, including syntactic dependency parsing, entity recognition, coreference resolution, and discourse parsing. We evaluate state-of-the-art NLP systems on GENTLE and find severe degradation for at least some genres in their performance on all tasks, which indicates GENTLE's utility as an evaluation dataset for NLP systems.

Key facts. Cited by 19 papers in the map (data through 14 Oct 2026). Builds on 16 of its 37 references that are in the map. First posted on arXiv 3 Jun 2023.

Builds on

16 of 37 references.

References · 16

  1. 2020Stanza: A Python Natural Language Processing Toolkit for Many Human Languagescited by 723
  2. 2021CUAD: An Expert-Annotated NLP Dataset for Legal Contract Reviewcited by 236
  3. 2021Trankit: A Light-Weight Transformer-based Toolkit for Multilingual Natural Language Processingcited by 68
  4. 2021A Sequence-to-Set Network for Nested Named Entity Recognitioncited by 40
  5. 2016What to do about non-standard (or non-canonical) language in NLPcited by 35
  6. 2018Extending a Parser to Distant Domains Using a Few Dozen Partially Annotated Examplescited by 34
  7. 2017Generalisation in Named Entity Recognition: A Quantitative Analysiscited by 25
  8. 2021OntoGUM: Evaluating Contextualized SOTA Coreference Resolution on 12 More Genrescited by 18
  9. 2021DisCoDisCo at the DISRPT2021 Shared Task: A System for Discourse Segmentation, Classification, and Connective Detectioncited by 15
  10. 2023Why Can't Discourse Parsing Generalize? A Thorough Investigation of the Impact of Data Diversitycited by 15
  11. 2020Treebanking User-Generated Content: a UD Based Overview of Guidelines, Corpora and Unified Recommendationscited by 13
  12. 2017Lexical Features in Coreference Resolution: To be Used With Cautioncited by 12
4 more
  1. 2019All Roads Lead to UD: Converting Stanford and Penn Parses to English Universal Dependencies with Multilayer Annotationscited by 9
  2. 2020A Cross-Genre Ensemble Approach to Robust Reddit Part of Speech Taggingcited by 9
  3. 2021WikiGUM: Exhaustive Entity Linking for Wikification in 12 Genrescited by 9
  4. 2022Out-of-Domain Evaluation of Finnish Dependency Parsingcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 5; 2025: 8; 2026: 5.

19 papers.

Sort:

  1. 2024eRST: A Signaled Graph Theory of Discourse Relations and Organizationcited by 14
  2. 2025GUM-SAGE: A Novel Dataset and Approach for Graded Entity Salience Predictioncited by 6
  3. 2024GDTB: Genre Diverse Data for English Shallow Discourse Parsing across Modalities, Text Types, and Domainscited by 5
  4. 2025Subjectivity in the Annotation of Bridging Anaphoracited by 4
  5. 2024AUTALIC: A Dataset for Anti-AUTistic Ableist Language In Contextcited by 3
  6. 2025Bridging Discourse Treebanks with a Unified Rhetorical Structure Parsercited by 3
  7. 2023What's Hard in English RST Parsing? Predictive Models for Error Analysiscited by 2
  8. 2024Unifying the Scope of Bridging Anaphora Types in English: Bridging Annotations in ARRAU and GUMcited by 2
  9. 2026Not Worth Mentioning? A Pilot Study on Salient Proposition Annotationcited by 2
  10. 2025DeDisCo at the DISRPT 2025 Shared Task: A System for Discourse Relation Classificationcited by 1
  11. 2025GUMBridge: a Corpus for Varieties of Bridging Anaphoracited by 1
  12. 2026Evaluation of Large Language Models in Legal Applications: Challenges, Methods, and Future Directionscited by 1
7 more
  1. 2024SPLICE: A Singleton-Enhanced PipeLIne for Coreference REsolutioncited by 0
  2. 2025Unpacking Ambiguity: The Interaction of Polysemous Discourse Markers and Non-DM Signalscited by 0
  3. 2025What makes an entity salient in discourse?cited by 0
  4. 2025CLaC at DISRPT 2025: Hierarchical Adapters for Cross-Framework Multi-lingual Discourse Relation Classificationcited by 0
  5. 2026DiscoExplorer: An Open Interface for the Study of Multilingual Discourse Relationscited by 0
  6. 2026Calibrated Typed Decisions for Entity, Relation, and Event Extractioncited by 0
  7. 2026Everyone Was Right: Lexical Representations in Language Models Are Both Abstract and Holisticcited by 0