# Inaccessible Neural Language Models Could Reinvigorate Linguistic Nativism

arXiv: https://arxiv.org/abs/2301.05272 · HTML: https://mlsanity.com/paper/2301.05272/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 18 of its 56 references that are in the map.
- First posted on arXiv 12 Jan 2023.

## Builds on (most cited first, 18 of 18)

- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models](https://mlsanity.com/paper/2206.04615/) (2022)
- [A Systematic Evaluation of Large Language Models of Code](https://mlsanity.com/paper/2202.13169/) (2022)
- [OpenNMT: Open-source Toolkit for Neural Machine Translation](https://mlsanity.com/paper/1709.03815/) (2017)
- [Deep Learning: A Critical Appraisal](https://mlsanity.com/paper/1801.00631/) (2018)
- [Large Pre-trained Language Models Contain Human-like Biases of What is Right and Wrong to Do](https://mlsanity.com/paper/2103.11790/) (2021)
- [A Restricted Black-box Adversarial Framework Towards Attacking Graph Embedding Models](https://mlsanity.com/paper/1908.01297/) (2019)
- [Leakage and the Reproducibility Crisis in ML-based Science](https://mlsanity.com/paper/2207.07048/) (2022)
- [COVID-ResNet: A Deep Learning Framework for Screening of COVID19 from Radiographs](https://mlsanity.com/paper/2003.14395/) (2020)
- [Arabic Text Diacritization Using Deep Neural Networks](https://mlsanity.com/paper/1905.01965/) (2019)
- [AI Ethics Statements -- Analysis and lessons learnt from NeurIPS Broader Impact Statements](https://mlsanity.com/paper/2111.01705/) (2021)
- [Eight Years of Face Recognition Research: Reproducibility, Achievements and Open Issues](https://mlsanity.com/paper/2208.04040/) (2022)
- [Automatic Tag Recommendation for Painting Artworks Using Diachronic Descriptions](https://mlsanity.com/paper/2004.09710/) (2020)
- [NusaCrowd: A Call for Open and Reproducible NLP Research in Indonesian Languages](https://mlsanity.com/paper/2207.10524/) (2022)
- [No Pattern, No Recognition: a Survey about Reproducibility and Distortion Issues of Text Clustering and Topic Modeling](https://mlsanity.com/paper/2208.01712/) (2022)
- [FreeTransfer-X: Safe and Label-Free Cross-Lingual Transfer from Off-the-Shelf Models](https://mlsanity.com/paper/2206.06586/) (2022)
- [Productive Reproducible Workflows for DNNs: A Case Study for Industrial Defect Detection](https://mlsanity.com/paper/2206.09359/) (2022)
- [Long-term Reproducibility for Neural Architecture Search](https://mlsanity.com/paper/2207.04821/) (2022)

## Abstract (arXiv)

Large Language Models (LLMs) have been making big waves in the machine learning community within the past few years. The impressive scalability of LLMs due to the advent of deep learning can be seen as a continuation of empiricist lingusitic methods, as opposed to rule-based linguistic methods that are grounded in a nativist perspective. Current LLMs are generally inaccessible to resource-constrained researchers, due to a variety of factors including closed source code. This work argues that this lack of accessibility could instill a nativist bias in researchers new to computational linguistics, given that new researchers may only have rule-based, nativist approaches to study to produce new work. Also, given that there are numerous critics of deep learning claiming that LLMs and related methods may soon lose their relevancy, we speculate that such an event could trigger a new wave of nativism in the language processing community. To prevent such a dramatic shift and placing favor in hybrid methods of rules and deep learning, we call upon researchers to open source their LLM code wherever possible to allow both empircist and hybrid approaches to remain accessible.