# Language Models Use Monotonicity to Assess NPI Licensing

arXiv: https://arxiv.org/abs/2105.13818 · HTML: https://mlsanity.com/paper/2105.13818/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 49 references that are in the map.
- First posted on arXiv 28 May 2021.

## Builds on (most cited first, 23 of 23)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [A Primer in BERTology: What we know about how BERT works](https://mlsanity.com/paper/2002.12327/) (2020)
- [Designing and Interpreting Probes with Control Tasks](https://mlsanity.com/paper/1909.03368/) (2019)
- [BLiMP: The Benchmark of Linguistic Minimal Pairs for English](https://mlsanity.com/paper/1912.00582/) (2019)
- [Assessing the Ability of LSTMs to Learn Syntax-Sensitive Dependencies](https://mlsanity.com/paper/1611.01368/) (2016)
- [What BERT is not: Lessons from a new suite of psycholinguistic diagnostics for language models](https://mlsanity.com/paper/1907.13528/) (2019)
- [Analysis Methods in Neural Language Processing: A Survey](https://mlsanity.com/paper/1812.08951/) (2018)
- [Targeted Syntactic Evaluation of Language Models](https://mlsanity.com/paper/1808.09031/) (2018)
- [Information-Theoretic Probing with Minimum Description Length](https://mlsanity.com/paper/2003.12298/) (2020)
- [Colorless green recurrent networks dream hierarchically](https://mlsanity.com/paper/1803.11138/) (2018)
- [A Systematic Assessment of Syntactic Generalization in Neural Language Models](https://mlsanity.com/paper/2005.03692/) (2020)
- [Neural Language Models as Psycholinguistic Subjects: Representations of Syntactic State](https://mlsanity.com/paper/1903.03260/) (2019)
- [Under the Hood: Using Diagnostic Classifiers to Investigate and Improve how Language Models Track Agreement Information](https://mlsanity.com/paper/1808.08079/) (2018)
- [The emergence of number and syntax units in LSTM language models](https://mlsanity.com/paper/1903.07435/) (2019)
- [Investigating BERT's Knowledge of Language: Five Analysis Methods with NPIs](https://mlsanity.com/paper/1909.02597/) (2019)
- [Do Language Models Understand Anything? On the Ability of LSTMs to Understand Negative Polarity Items](https://mlsanity.com/paper/1808.10627/) (2018)
- [Can neural networks acquire a structural bias from raw linguistic data?](https://mlsanity.com/paper/2007.06761/) (2020)
- [Structural Supervision Improves Learning of Non-Local Grammatical Dependencies](https://mlsanity.com/paper/1903.00943/) (2019)
- [Analyzing and Interpreting Neural Networks for NLP: A Report on the First BlackboxNLP Workshop](https://mlsanity.com/paper/1904.04063/) (2019)
- [Infusing Finetuning with Semantic Dependencies](https://mlsanity.com/paper/2012.05395/) (2020)
- [Language Modelling as a Multi-Task Problem](https://mlsanity.com/paper/2101.11287/) (2021)
- [diagNNose: A Library for Neural Activation Analysis](https://mlsanity.com/paper/2011.06819/) (2020)

## Built on by (most cited first, 30 of 31)

- [State-of-the-art generalisation research in NLP: A taxonomy and review](https://mlsanity.com/paper/2210.03050/) (2022)
- [Language models are not naysayers: An analysis of language models on negation benchmarks](https://mlsanity.com/paper/2306.08189/) (2023)
- [Language Models Learn Rare Phenomena from Less Rare Phenomena: The Case of the Missing AANNs](https://mlsanity.com/paper/2403.19827/) (2024)
- [Structural Persistence in Language Models: Priming as a Window into Abstract Language Representations](https://mlsanity.com/paper/2109.14989/) (2021)
- [Filtered Corpus Training (FiCT) Shows that Language Models can Generalize from Indirect Evidence](https://mlsanity.com/paper/2405.15750/) (2024)
- [Sparse Interventions in Language Models with Differentiable Masking](https://mlsanity.com/paper/2112.06837/) (2021)
- [Can Language Models Learn Typologically Implausible Languages?](https://mlsanity.com/paper/2502.12317/) (2025)
- [DecoderLens: Layerwise Interpretation of Encoder-Decoder Transformers](https://mlsanity.com/paper/2310.03686/) (2023)
- [Both Direct and Indirect Evidence Contribute to Dative Alternation Preferences in Language Models](https://mlsanity.com/paper/2503.20850/) (2025)
- [Generalizations across filler-gap dependencies in neural language models](https://mlsanity.com/paper/2410.18225/) (2024)
- [Can Language Models Be Tricked by Language Illusions? Easier with Syntax, Harder with Semantics](https://mlsanity.com/paper/2311.01386/) (2023)
- [A systematic framework for generating novel experimental hypotheses from language models](https://mlsanity.com/paper/2408.05086/) (2024)
- [Causal Transformers Perform Below Chance on Recursive Nested Constructions, Unlike Humans](https://mlsanity.com/paper/2110.07240/) (2021)
- [Simple Linguistic Inferences of Large Language Models (LLMs): Blind Spots and Blinds](https://mlsanity.com/paper/2305.14785/) (2023)
- [From Form(s) to Meaning: Probing the Semantic Depths of Language Models Using Multisense Consistency](https://mlsanity.com/paper/2404.12145/) (2024)
- [Probing LLMs for Multilingual Discourse Generalization Through a Unified Label Set](https://mlsanity.com/paper/2503.10515/) (2025)
- [Testing Pre-trained Language Models' Understanding of Distributivity via Causal Mediation Analysis](https://mlsanity.com/paper/2209.04761/) (2022)
- [Language Models at the Syntax-Semantics Interface: A Case Study of the Long-Distance Binding of Chinese Reflexive ziji](https://mlsanity.com/paper/2504.02116/) (2025)
- [What Exactly do Children Receive in Language Acquisition? A Case Study on CHILDES with Automated Detection of Filler-Gap Dependencies](https://mlsanity.com/paper/2603.02082/) (2026)
- [Cross-Modal Taxonomic Generalization in (Vision-) Language Models](https://mlsanity.com/paper/2603.07474/) (2026)
- [Transformers in the loop: Polarity in neural models of language](https://mlsanity.com/paper/2109.03926/) (2021)
- [Connecting degree and polarity: An artificial language learning study](https://mlsanity.com/paper/2109.06333/) (2021)
- [Black Big Boxes: Tracing Adjective Order Preferences in Large Language Models](https://mlsanity.com/paper/2407.02136/) (2024)
- [Investigating Syntactic Biases in Multilingual Transformers with RC Attachment Ambiguities in Italian and English](https://mlsanity.com/paper/2504.09886/) (2025)
- [Uncovering Implicit Bias in Large Language Models with Concept Learning Dataset](https://mlsanity.com/paper/2510.01219/) (2025)
- [Spectral Archaeology: The Causal Topology of Model Evolution](https://mlsanity.com/paper/2601.03424/) (2026)
- [Fine-Grained Analysis of Shared Syntactic Mechanisms in Language Models](https://mlsanity.com/paper/2604.22166/) (2026)
- [Targeted Linguistic Analysis of Sign Language Models with Minimal Translation Pairs](https://mlsanity.com/paper/2604.27232/) (2026)
- [Language Models Generalize to Human-like Word Order Preferences](https://mlsanity.com/paper/2608.05028/) (2026)
- [Disentangling Statistical Preemption from Entrenchment in Language Models' Avoidance of Overgeneralization](https://mlsanity.com/paper/2609.01794/) (2026)

## Abstract (arXiv)

We investigate the semantic knowledge of language models (LMs), focusing on (1) whether these LMs create categories of linguistic environments based on their semantic monotonicity properties, and (2) whether these categories play a similar role in LMs as in human language understanding, using negative polarity item licensing as a case study. We introduce a series of experiments consisting of probing with diagnostic classifiers (DCs), linguistic acceptability tasks, as well as a novel DC ranking method that tightly connects the probing results to the inner workings of the LM. By applying our experimental pipeline to LMs trained on various filtered corpora, we are able to gain stronger insights into the semantic generalizations that are acquired by these models.