# Community size rather than grammatical complexity better predicts Large Language Model accuracy in a novel Wug Test

arXiv: https://arxiv.org/abs/2510.12463 · HTML: https://mlsanity.com/paper/2510.12463/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 51 references that are in the map.
- First posted on arXiv 14 Oct 2025.

## Builds on (most cited first, 16 of 16)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [A Comprehensive Overview of Large Language Models](https://mlsanity.com/paper/2307.06435/) (2023)
- [Large Language Models Struggle to Learn Long-Tail Knowledge](https://mlsanity.com/paper/2211.08411/) (2022)
- [Persistent Anti-Muslim Bias in Large Language Models](https://mlsanity.com/paper/2101.05783/) (2021)
- [On Second Thought, Let's Not Think Step by Step! Bias and Toxicity in Zero-Shot Reasoning](https://mlsanity.com/paper/2212.08061/) (2022)
- [State-of-the-art generalisation research in NLP: A taxonomy and review](https://mlsanity.com/paper/2210.03050/) (2022)
- [Testing AI on language comprehension tasks reveals insensitivity to underlying meaning](https://mlsanity.com/paper/2302.12313/) (2023)
- [A Sentence is Worth a Thousand Pictures: Can Large Language Models Understand Hum4n L4ngu4ge and the W0rld behind W0rds?](https://mlsanity.com/paper/2308.00109/) (2023)
- [LLMs as Models for Analogical Reasoning](https://mlsanity.com/paper/2406.13803/) (2024)
- [Evaluating Neural Language Models as Cognitive Models of Language Acquisition](https://mlsanity.com/paper/2310.20093/) (2023)
- [Language in Vivo vs. in Silico: Size Matters but Larger Language Models Still Do Not Comprehend Language on a Par with Humans Due to Impenetrable Semantic Reference](https://mlsanity.com/paper/2404.14883/) (2024)
- [Tokenization and Morphology in Multilingual Language Models: A Comparative Analysis of mT5 and ByT5](https://mlsanity.com/paper/2410.11627/) (2024)
- [Counting the Bugs in ChatGPT's Wugs: A Multilingual Investigation into the Morphological Capabilities of a Large Language Model](https://mlsanity.com/paper/2310.15113/) (2023)
- [Emergence of a phonological bias in ChatGPT](https://mlsanity.com/paper/2305.15929/) (2023)
- [How Well Do Large Language Models Understand Syntax? An Evaluation by Asking Natural Language Questions](https://mlsanity.com/paper/2311.08287/) (2023)

## Built on by (most cited first, 2 of 2)

- [Tracing the ongoing emergence of human-like reasoning in Large Language Models](https://mlsanity.com/paper/2605.21299/) (2026)
- [What's in a Name? Morphological Shortcuts by LLMs in Pharmacology](https://mlsanity.com/paper/2606.05616/) (2026)

## Abstract (arXiv)

The linguistic abilities of Large Language Models are a matter of ongoing debate. This study contributes to this discussion by investigating model performance in a morphological generalization task that involves novel words. Using a multilingual adaptation of the Wug Test, six models were tested across four partially unrelated languages (Catalan, English, Greek, and Spanish) and compared with human speakers. The aim is to determine whether model accuracy approximates human competence and whether it is shaped primarily by linguistic complexity or by the size of the linguistic community, which affects the quantity of available training data. Consistent with previous research, the results show that the models are able to generalize morphological processes to unseen words with human-like accuracy. However, accuracy patterns align more closely with community size and data availability than with structural complexity, refining earlier claims in the literature. In particular, languages with larger speaker communities and stronger digital representation, such as Spanish and English, revealed higher accuracy than less-resourced ones like Catalan and Greek. Overall, our findings suggest that model behavior is mainly driven by the richness of linguistic resources rather than by sensitivity to grammatical complexity, reflecting a form of performance that resembles human linguistic competence only superficially.