# How broad is that claim? Mapping Generalisation in NLP Research

arXiv: https://arxiv.org/abs/2609.14770 · HTML: https://mlsanity.com/paper/2609.14770/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 8 of its 49 references that are in the map.
- First posted on arXiv 13 Sep 2026.

## Builds on (most cited first, 8 of 8)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [SciRepEval: A Multi-Format Benchmark for Scientific Document Representations](https://mlsanity.com/paper/2211.13308/) (2022)
- [NLPeer: A Unified Resource for the Computational Study of Peer Review](https://mlsanity.com/paper/2211.06651/) (2022)
- [GenericsKB: A Knowledge Base of Generic Statements](https://mlsanity.com/paper/2005.00660/) (2020)
- [Decomposing Generalization: Models of Generic, Habitual, and Episodic Statements](https://mlsanity.com/paper/1901.11429/) (2019)
- [MGen: Millions of Naturally Occurring Generics in Context](https://mlsanity.com/paper/2509.26160/) (2025)
- [Generics in science communication: Misaligned interpretations across laypeople, scientists, and large language models](https://mlsanity.com/paper/2602.06190/) (2026)

## Abstract (arXiv)

Generalisations are common in scientific communication, even though they are semantically ambiguous. An automated method is needed to identify and categorise claims according to their level of generalisation, in order help detect an over-reliance on generalisations and possible misrepresentations of scientific findings. We introduce a comprehensive taxonomy of generalisations in the scientific domain, NLPGenX, which labels claims according to their level of generality and framing within the text. We operationalise this taxonomy with an LLM-powered framework, NLPGenA, that automatically classifies sentences from scientific articles into 5 different generalisation classes. We validate our framework with human annotators and use the framework to construct a large-scale dataset of NLP papers annotated according to generality, with auxiliary labels for hedging and vague descriptors (NLPGens). We use NLPGens to analyse the use of generalisations in NLP papers across multiple venues and subdomains, and to examine associations with citation counts, hedging, and vague descriptors.