# Is deeper always better? Replacing linear mappings with deep learning networks in the Discriminative Lexicon Model

arXiv: https://arxiv.org/abs/2410.04259 · HTML: https://mlsanity.com/paper/2410.04259/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 77 references that are in the map.
- First posted on arXiv 5 Oct 2024.

## Builds on (most cited first, 15 of 15)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [Learning Word Vectors for 157 Languages](https://mlsanity.com/paper/1802.06893/) (2018)
- [In-Context Impersonation Reveals Large Language Models' Strengths and Biases](https://mlsanity.com/paper/2305.14930/) (2023)
- [Syntactic Structure from Deep Learning](https://mlsanity.com/paper/2004.10827/) (2020)
- [Recurrent Neural Networks in Linguistic Theory: Revisiting Pinker and Prince (1988) and the Past Tense Debate](https://mlsanity.com/paper/1807.04783/) (2018)
- [Are we there yet? Encoder-decoder neural networks as cognitive models of English past tense inflection](https://mlsanity.com/paper/1906.01280/) (2019)
- [Inflecting when there's no majority: Limitations of encoder-decoder neural networks as cognitive models for German plurals](https://mlsanity.com/paper/2005.08826/) (2020)
- [Word-specific tonal realizations in Mandarin](https://mlsanity.com/paper/2405.07006/) (2024)
- [Learning Zero-Shot Multifaceted Visually Grounded Word Embeddings via Multi-Task Training](https://mlsanity.com/paper/2104.07500/) (2021)
- [Vector Space Morphology with Linear Discriminative Learning](https://mlsanity.com/paper/2107.03950/) (2021)
- [How trial-to-trial learning shapes mappings in the mental lexicon: Modelling Lexical Decision with Linear Discriminative Learning](https://mlsanity.com/paper/2207.00430/) (2022)
- [Frequency effects in Linear Discriminative Learning](https://mlsanity.com/paper/2306.11044/) (2023)
- [Semantic properties of English nominal pluralization: Insights from word embeddings](https://mlsanity.com/paper/2203.15424/) (2022)

## Built on by (most cited first, 4 of 4)

- [The realization of tones in spontaneous spoken Taiwan Mandarin: a corpus-based survey and theory-driven computational modeling](https://mlsanity.com/paper/2503.23163/) (2025)
- [Frequency matters: Modeling irregular morphological patterns in Spanish with Transformers](https://mlsanity.com/paper/2410.21013/) (2024)
- [Transformers over-extend what humans underlearn: the case of Spanish L-shaped morphome](https://mlsanity.com/paper/2507.21556/) (2025)
- [Compound interpretation is based on analogy](https://mlsanity.com/paper/2610.01688/) (2026)

## Abstract (arXiv)

Recently, deep learning models have increasingly been used in cognitive modelling of language. This study asks whether deep learning can help us to better understand the learning problem that needs to be solved by speakers, above and beyond linear methods. We utilise the Discriminative Lexicon Model introduced by Baayen and colleagues, which models comprehension and production with mappings between numeric form and meaning vectors. While so far, these mappings have been linear (Linear Discriminative Learning, LDL), in the present study we replace them with deep dense neural networks (Deep Discriminative Learning, DDL). We find that DDL affords more accurate mappings for large and diverse datasets from English and Dutch, but not necessarily for Estonian and Taiwan Mandarin. DDL outperforms LDL in particular for words with pseudo-morphological structure such as chol+er. Applied to average reaction times, we find that DDL is outperformed by frequency-informed linear mappings (FIL). However, DDL trained in a frequency-informed way ('frequency-informed' deep learning, FIDDL) substantially outperforms FIL. Finally, while linear mappings can very effectively be updated from trial-to-trial to model incremental lexical learning, deep mappings cannot do so as effectively. At present, both linear and deep mappings are informative for understanding language.