# Differences in Typological Alignment in Language Models' Treatment of Differential Argument Marking

arXiv: https://arxiv.org/abs/2602.17653 · HTML: https://mlsanity.com/paper/2602.17653/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 54 references that are in the map.
- First posted on arXiv 19 Feb 2026.

## Builds on (most cited first, 16 of 16)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Probing Classifiers: Promises, Shortcomings, and Advances](https://mlsanity.com/paper/2102.12452/) (2021)
- [Designing and Interpreting Probes with Control Tasks](https://mlsanity.com/paper/1909.03368/) (2019)
- [BLiMP: The Benchmark of Linguistic Minimal Pairs for English](https://mlsanity.com/paper/1912.00582/) (2019)
- [Large Language Models for Data Annotation and Synthesis: A Survey](https://mlsanity.com/paper/2402.13446/) (2024)
- [Constituency Parsing with a Self-Attentive Encoder](https://mlsanity.com/paper/1805.01052/) (2018)
- [Embers of Autoregression: Understanding Large Language Models Through the Problem They are Trained to Solve](https://mlsanity.com/paper/2309.13638/) (2023)
- [How Linguistics Learned to Stop Worrying and Love the Language Models](https://mlsanity.com/paper/2501.17047/) (2025)
- [Examining the Inductive Bias of Neural Language Models with Artificial Languages](https://mlsanity.com/paper/2106.01044/) (2021)
- [Filtered Corpus Training (FiCT) Shows that Language Models can Generalize from Indirect Evidence](https://mlsanity.com/paper/2405.15750/) (2024)
- [Can Language Models Learn Typologically Implausible Languages?](https://mlsanity.com/paper/2502.12317/) (2025)
- [Information Locality as an Inductive Bias for Neural Language Models](https://mlsanity.com/paper/2506.05136/) (2025)
- [Manipulating language models' training data to study syntactic constraint learning: the case of English passivization](https://mlsanity.com/paper/2407.04593/) (2024)
- [Emergent Word Order Universals from Cognitively-Motivated Language Models](https://mlsanity.com/paper/2402.12363/) (2024)
- [Simulating the Emergence of Differential Case Marking with Communicating Neural-Network Agents](https://mlsanity.com/paper/2502.04038/) (2025)
- [Is Structure Dependence Shaped for Efficient Communication?: A Case Study on Coordination](https://mlsanity.com/paper/2410.10556/) (2024)

## Built on by (most cited first, 1 of 1)

- [Language Models Generalize to Human-like Word Order Preferences](https://mlsanity.com/paper/2608.05028/) (2026)

## Abstract (arXiv)

Recent work has shown that language models (LMs) trained on synthetic corpora can exhibit typological preferences that resemble cross-linguistic regularities in human languages, particularly for syntactic phenomena such as word order. In this paper, we extend this paradigm to differential argument marking (DAM), a semantic licensing system in which morphological marking depends on semantic prominence. Using a controlled synthetic learning method, we train GPT-2 models on 18 corpora implementing distinct DAM systems and evaluate their generalization using minimal pairs. Our results reveal a dissociation between two typological dimensions of DAM. Models reliably exhibit human-like preferences for natural markedness direction, favoring systems in which overt marking targets semantically atypical arguments. In contrast, models do not reproduce the strong object preference in human languages, in which overt marking in DAM more often targets objects rather than subjects. These findings suggest that different typological tendencies may arise from distinct underlying sources.