# Native Language Identification with Large Language Models

arXiv: https://arxiv.org/abs/2312.07819 · HTML: https://mlsanity.com/paper/2312.07819/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 2 of its 20 references that are in the map.
- First posted on arXiv 13 Dec 2023.

## Builds on (most cited first, 2 of 2)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Scaling Native Language Identification with Transformer Adapters](https://mlsanity.com/paper/2211.10117/) (2022)

## Built on by (most cited first, 1 of 1)

- [From Feature-Based Models to Generative AI: Validity Evidence for Constructed Response Scoring](https://mlsanity.com/paper/2603.19280/) (2026)

## Abstract (arXiv)

We present the first experiments on Native Language Identification (NLI) using LLMs such as GPT-4. NLI is the task of predicting a writer's first language by analyzing their writings in a second language, and is used in second language acquisition and forensic linguistics. Our results show that GPT models are proficient at NLI classification, with GPT-4 setting a new performance record of 91.7% on the benchmark TOEFL11 test set in a zero-shot setting. We also show that unlike previous fully-supervised settings, LLMs can perform NLI without being limited to a set of known classes, which has practical implications for real-world applications. Finally, we also show that LLMs can provide justification for their choices, providing reasoning based on spelling errors, syntactic patterns, and usage of directly translated linguistic patterns.