# Systematicity in GPT-3's Interpretation of Novel English Noun Compounds

arXiv: https://arxiv.org/abs/2210.09492 · HTML: https://mlsanity.com/paper/2210.09492/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 25 references that are in the map.
- First posted on arXiv 18 Oct 2022.

## Builds on (most cited first, 6 of 6)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Texygen: A Benchmarking Platform for Text Generation Models](https://mlsanity.com/paper/1802.01886/) (2018)
- [It's not Rocket Science : Interpreting Figurative Language in Narratives](https://mlsanity.com/paper/2109.00087/) (2021)
- [AmbiPun: Generating Humorous Puns with Ambiguous Context](https://mlsanity.com/paper/2205.01825/) (2022)
- [Will it Unblend?](https://mlsanity.com/paper/2009.09123/) (2020)
- [Characterizing Idioms: Conventionality and Contingency](https://mlsanity.com/paper/2104.08664/) (2021)

## Built on by (most cited first, 9 of 9)

- [From chocolate bunny to chocolate crocodile: Do Language Models Understand Noun Compounds?](https://mlsanity.com/paper/2305.10568/) (2023)
- [Rolling the DICE on Idiomaticity: How LLMs Fail to Grasp Context](https://mlsanity.com/paper/2410.16069/) (2024)
- [Semantics of Multiword Expressions in Transformer-Based Models: A Survey](https://mlsanity.com/paper/2401.15393/) (2024)
- [Memorization or Reasoning? Exploring the Idiom Understanding of LLMs](https://mlsanity.com/paper/2505.16216/) (2025)
- [The Inner Sentiments of a Thought](https://mlsanity.com/paper/2307.01784/) (2023)
- [Deep Natural Language Feature Learning for Interpretable Prediction](https://mlsanity.com/paper/2311.05754/) (2023)
- [Systematic word meta-sense extension](https://mlsanity.com/paper/2311.13029/) (2023)
- [Willkommens-Merkel, Chaos-Johnson, and Tore-Klose: Modeling the Evaluative Meaning of German Personal Name Compounds](https://mlsanity.com/paper/2404.04031/) (2024)
- [Probing BERT for German Compound Semantics](https://mlsanity.com/paper/2505.14130/) (2025)

## Abstract (arXiv)

Levin et al. (2019) show experimentally that the interpretations of novel English noun compounds (e.g., stew skillet), while not fully compositional, are highly predictable based on whether the modifier and head refer to artifacts or natural kinds. Is the large language model GPT-3 governed by the same interpretive principles? To address this question, we first compare Levin et al.'s experimental data with GPT-3 generations, finding a high degree of similarity. However, this evidence is consistent with GPT3 reasoning only about specific lexical items rather than the more abstract conceptual categories of Levin et al.'s theory. To probe more deeply, we construct prompts that require the relevant kind of conceptual reasoning. Here, we fail to find convincing evidence that GPT-3 is reasoning about more than just individual lexical items. These results highlight the importance of controlling for low-level distributional regularities when assessing whether a large language model latently encodes a deeper theory.