# Can LSTM Learn to Capture Agreement? The Case of Basque

arXiv: https://arxiv.org/abs/1809.04022 · HTML: https://mlsanity.com/paper/1809.04022/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Sep 2018.

## Built on by (most cited first, 17 of 17)

- [Multi-VALUE: A Framework for Cross-Dialectal English NLP](https://mlsanity.com/paper/2212.08011/) (2022)
- [CLiMP: A Benchmark for Chinese Language Model Evaluation](https://mlsanity.com/paper/2101.11131/) (2021)
- [Square One Bias in NLP: Towards a Multi-Dimensional Exploration of the Research Manifold](https://mlsanity.com/paper/2206.09755/) (2022)
- [Dissociating language and thought in large language models](https://mlsanity.com/paper/2301.06627/) (2023)
- [How to Plant Trees in Language Models: Data and Architectural Effects on the Emergence of Syntactic Inductive Biases](https://mlsanity.com/paper/2305.19905/) (2023)
- [UA-GEC: Grammatical Error Correction and Fluency Corpus for the Ukrainian Language](https://mlsanity.com/paper/2103.16997/) (2021)
- [JCoLA: Japanese Corpus of Linguistic Acceptability](https://mlsanity.com/paper/2309.12676/) (2023)
- [Controlled Evaluation of Syntactic Knowledge in Multilingual Language Models](https://mlsanity.com/paper/2411.07474/) (2024)
- [Are word boundaries useful for unsupervised language learning?](https://mlsanity.com/paper/2210.02956/) (2022)
- [Schr"odinger's Tree -- On Syntax and Neural Language Models](https://mlsanity.com/paper/2110.08887/) (2021)
- [Empowering Cross-lingual Behavioral Testing of NLP Models with Typological Features](https://mlsanity.com/paper/2307.05454/) (2023)
- [Morph Call: Probing Morphosyntactic Content of Multilingual Transformers](https://mlsanity.com/paper/2104.12847/) (2021)
- [Uncovering Constraint-Based Behavior in Neural Models via Targeted Fine-Tuning](https://mlsanity.com/paper/2106.01207/) (2021)
- [Controlled Evaluation of Grammatical Knowledge in Mandarin Chinese Language Models](https://mlsanity.com/paper/2109.11058/) (2021)
- [Which Word Orders Facilitate Length Generalization in LMs? An Investigation with GCG-Based Artificial Languages](https://mlsanity.com/paper/2510.12722/) (2025)
- [Towards Zero-shot Language Modeling](https://mlsanity.com/paper/2108.03334/) (2021)
- [Multilingual Syntax-aware Language Modeling through Dependency Tree Conversion](https://mlsanity.com/paper/2204.08644/) (2022)

## Abstract (arXiv)

Sequential neural networks models are powerful tools in a variety of Natural Language Processing (NLP) tasks. The sequential nature of these models raises the questions: to what extent can these models implicitly learn hierarchical structures typical to human language, and what kind of grammatical phenomena can they acquire? We focus on the task of agreement prediction in Basque, as a case study for a task that requires implicit understanding of sentence structure and the acquisition of a complex but consistent morphological system. Analyzing experimental results from two syntactic prediction tasks -- verb number prediction and suffix recovery -- we find that sequential models perform worse on agreement prediction in Basque than one might expect on the basis of a previous agreement prediction work in English. Tentative findings based on diagnostic classifiers suggest the network makes use of local heuristics as a proxy for the hierarchical structure of the sentence. We propose the Basque agreement prediction task as challenging benchmark for models that attempt to learn regularities in human language.