Differences in Typological Alignment in Language Models' Treatment of Differential Argument Marking
Not on the map. Why?
Abstract (arXiv)
Recent work has shown that language models (LMs) trained on synthetic corpora can exhibit typological preferences that resemble cross-linguistic regularities in human languages, particularly for syntactic phenomena such as word order. In this paper, we extend this paradigm to differential argument marking (DAM), a semantic licensing system in which morphological marking depends on semantic prominence. Using a controlled synthetic learning method, we train GPT-2 models on 18 corpora implementing distinct DAM systems and evaluate their generalization using minimal pairs. Our results reveal a dissociation between two typological dimensions of DAM. Models reliably exhibit human-like preferences for natural markedness direction, favoring systems in which overt marking targets semantically atypical arguments. In contrast, models do not reproduce the strong object preference in human languages, in which overt marking in DAM more often targets objects rather than subjects. These findings suggest that different typological tendencies may arise from distinct underlying sources.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 16 of its 54 references that are in the map. First posted on arXiv 19 Feb 2026.
Builds on
16 of 54 references.
References · 16
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2021Probing Classifiers: Promises, Shortcomings, and Advancescited by 831
- 2019Designing and Interpreting Probes with Control Taskscited by 608
- 2019BLiMP: The Benchmark of Linguistic Minimal Pairs for Englishcited by 553
- 2024Large Language Models for Data Annotation and Synthesis: A Surveycited by 201
- 2018Constituency Parsing with a Self-Attentive Encodercited by 196
- 2023Embers of Autoregression: Understanding Large Language Models Through the Problem They are Trained to Solvecited by 175
- 2025How Linguistics Learned to Stop Worrying and Love the Language Modelscited by 50
- 2021Examining the Inductive Bias of Neural Language Models with Artificial Languagescited by 40
- 2024Filtered Corpus Training (FiCT) Shows that Language Models can Generalize from Indirect Evidencecited by 35
- 2025Can Language Models Learn Typologically Implausible Languages?cited by 23
- 2025Information Locality as an Inductive Bias for Neural Language Modelscited by 14
4 more
- 2024Manipulating language models' training data to study syntactic constraint learning: the case of English passivizationcited by 12
- 2024Emergent Word Order Universals from Cognitively-Motivated Language Modelscited by 11
- 2025Simulating the Emergence of Differential Case Marking with Communicating Neural-Network Agentscited by 5
- 2024Is Structure Dependence Shaped for Efficient Communication?: A Case Study on Coordinationcited by 1
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026Typological Alignment of Stack-Based Language Models on Mildly Context-Sensitive Artificial Languages
- 2025Both Direct and Indirect Evidence Contribute to Dative Alternation Preferences in Language Models
- 2026LLM Alignment--Utility Asymmetry under Semantic-Preserving Transformations
- 2026Typological Feature Prediction with Large Language Models: An In-Context Learning Approach
- 2026Behavioral and Representational Evidence of Binomial Ordering Preferences in Large Language Models
- 2026Investigating Linguistic Steering: An Analysis of Adjectival Effects Across Large Language Model Architectures
- 2026Preferences for Idiomatic Language are Acquired Slowly -- and Forgotten Quickly: A Case Study on Swedish
- 2026Left-Branching Transformers Excel at Right-Branching Languages: Data Shapes Word Order Preferences in Language Models