mlsanityData to 14 Oct 2026

A Discerning Several Thousand Judgments: GPT-3 Rates the Article + Adjective + Numeral + Noun Construction

arXiv 2301.12564 ↗First posted 29 Jan 2023cs.CLcited by 22

Not on the map. Why?

Abstract (arXiv)

Knowledge of syntax includes knowledge of rare, idiosyncratic constructions. LLMs must overcome frequency biases in order to master such constructions. In this study, I prompt GPT-3 to give acceptability judgments on the English-language Article + Adjective + Numeral + Noun construction (e.g., "a lovely five days"). I validate the prompt using the CoLA corpus of acceptability judgments and then zero in on the AANN construction. I compare GPT- 3's judgments to crowdsourced human judgments on a subset of sentences. GPT-3's judgments are broadly similar to human judgments and generally align with proposed constraints in the literature but, in some cases, GPT-3's judgments and human judgments diverge from the literature and from each other.

Key facts. Cited by 22 papers in the map (data through 14 Oct 2026). Builds on 19 of its 40 references that are in the map. First posted on arXiv 29 Jan 2023.

Builds on

19 of 40 references.

References · 19

  1. 2019BERT Rediscovers the Classical NLP Pipelinecited by 1,059
  2. 2018Neural Network Acceptability Judgmentscited by 958
  3. 2019BLiMP: The Benchmark of Linguistic Minimal Pairs for Englishcited by 564
  4. 2021Causal Abstractions of Neural Networkscited by 422
  5. 2016Assessing the Ability of LSTMs to Learn Syntax-Sensitive Dependenciescited by 354
  6. 2019What BERT is not: Lessons from a new suite of psycholinguistic diagnostics for language modelscited by 320
  7. 2018Colorless green recurrent networks dream hierarchicallycited by 213
  8. 2019Neural Language Models as Psycholinguistic Subjects: Representations of Syntactic Statecited by 138
  9. 2020Syntactic Structure from Deep Learningcited by 99
  10. 2022What Artificial Neural Networks Can Tell Us About Human Language Acquisitioncited by 99
  11. 2021Counterfactual Interventions Reveal the Causal Effect of Relative Clause Representations on Agreement Predictioncited by 59
  12. 2021On the proper role of linguistically-oriented deep net analysis in linguistic theorizingcited by 43
7 more
  1. 2020Can neural networks acquire a structural bias from raw linguistic data?cited by 35
  2. 2022The Better Your Syntax, the Better Your Semantics? Probing Pretrained Language Models for the English Comparative Correlativecited by 35
  3. 2020CxGBERT: BERT meets Construction Grammarcited by 32
  4. 2021A Targeted Assessment of Incremental Processing in Neural LanguageModels and Humanscited by 32
  5. 2022Neural reality of argument structure constructionscited by 25
  6. 2020Investigating representations of verb bias in neural language modelscited by 22
  7. 2020(Re)construing Meaning in NLPcited by 16

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 5; 2024: 5; 2025: 6; 2026: 6.

22 papers.

Sort:

By area:

  1. 2023Language Model Behavior: A Comprehensive Surveycited by 92
  2. 2024Language Models Learn Rare Phenomena from Less Rare Phenomena: The Case of the Missing AANNscited by 65
  3. 2024Beyond Preferences in AI Alignmentcited by 49
  4. 2023Do large language models resemble humans in language use?cited by 34
  5. 2025Language Models Fail to Introspect About Their Knowledge of Languagecited by 30
  6. 2023A Method for Studying Semantic Construal in Grammatical Constructions with Interpretable Contextual Embedding Spacescited by 13
  7. 2023AI, write an essay for me: A large-scale comparison of human-written versus ChatGPT-generated essayscited by 10
  8. 2024UCxn: Typologically Informed Annotation of Constructions Atop Universal Dependenciescited by 7
  9. 2025Constructions are Revealed in Word Distributionscited by 6
  10. 2024Hybrid Human-LLM Corpus Construction and LLM Evaluation for Rare Linguistic Phenomenacited by 5
  11. 2025Beyond Memorization: Assessing Semantic Generalization in Large Language Models Using Phrasal Constructionscited by 5
  12. 2025Construction Identification and Disambiguation Using BERT: A Case Study of NPNcited by 4
10 more
  1. 2025Unpacking Let Alone: Human-Scale Models Generalize to a Rare Construction in Form but not Meaningcited by 4
  2. 2023Syntax and Semantics Meet in the "Middle": Probing the Syntax-Semantics Interface of LMs Through Agentivitycited by 3
  3. 2024Analysis of Argument Structure Constructions in the Large Language Model BERTcited by 3
  4. 2025LLMs Learn Constructions That Humans Do Not Knowcited by 1
  5. 2026The Grammar of Transformers: A Systematic Review of Interpretability Research on Syntactic Knowledge in Language Modelscited by 1
  6. 2026Multilingual Large Language Models do not comprehend all natural languages to equal degreescited by 1
  7. 2026Tracing the ongoing emergence of human-like reasoning in Large Language Modelscited by 0
  8. 2026Language Models Learn Constructional Semantics, Not To Mention Syntax: Investigating LM Understanding of Paired-Focus Constructionscited by 0
  9. 2026Linguistic Productivity in Large Language Models: Models Coerce, but do not Preemptcited by 0
  10. 2026FreqBLiMP: Frequency-Controlled Minimal Pairs Reveal Robustness and Fragility of LLMs Under Lexical Raritycited by 0