mlsanityData to 14 Oct 2026

Second Language Acquisition of Neural Language Models

arXiv 2306.02920 ↗First posted 5 Jun 2023cs.CLcited by 8

Not on the map. Why?

Abstract (arXiv)

With the success of neural language models (LMs), their language acquisition has gained much attention. This work sheds light on the second language (L2) acquisition of LMs, while previous work has typically explored their first language (L1) acquisition. Specifically, we trained bilingual LMs with a scenario similar to human L2 acquisition and analyzed their cross-lingual transfer from linguistic perspectives. Our exploratory experiments demonstrated that the L1 pretraining accelerated their linguistic generalization in L2, and language transfer configurations (e.g., the L1 choice, and presence of parallel texts) substantially affected their generalizations. These clarify their (non-)human-like L2 acquisition in particular aspects.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 19 of its 43 references that are in the map. First posted on arXiv 5 Jun 2023.

Builds on

19 of 43 references.

References · 19

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,935
  3. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,918
  4. 2019Cross-lingual Language Model Pretrainingcited by 1,090
  5. 2019BLiMP: The Benchmark of Linguistic Minimal Pairs for Englishcited by 564
  6. 2019CCNet: Extracting High Quality Monolingual Datasets from Web Crawl Datacited by 548
  7. 2022Language Models are Multilingual Chain-of-Thought Reasonerscited by 530
  8. 2016Assessing the Ability of LSTMs to Learn Syntax-Sensitive Dependenciescited by 354
  9. 2022The Geometry of Multilingual Language Model Representationscited by 95
  10. 2021Probing Across Time: What Does RoBERTa Know and When?cited by 73
  11. 2020Cross-Linguistic Syntactic Evaluation of Word Prediction Modelscited by 55
  12. 2021When is BERT Multilingual? Isolating Crucial Ingredients for Cross-lingual Transfercited by 42
7 more
  1. 2020Can neural networks acquire a structural bias from raw linguistic data?cited by 35
  2. 2018Revisiting the poverty of the stimulus: hierarchical generalization without a hierarchical bias in recurrent neural networkscited by 33
  3. 2022Analyzing the Mono- and Cross-Lingual Pretraining Dynamics of Multilingual Language Modelscited by 32
  4. 2022Pretraining with Artificial Language: Studying Transferable Knowledge in Language Modelscited by 28
  5. 2018Recurrent Neural Networks in Linguistic Theory: Revisiting Pinker and Prince (1988) and the Past Tense Debatecited by 27
  6. 2020How Furiously Can Colourless Green Ideas Sleep? Sentence Acceptability in Contextcited by 26
  7. 2021Assessing the Syntactic Capabilities of Transformer-based Multilingual Language Modelscited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 2; 2026: 5.

8 papers.

  1. 2024Investigating Critical Period Effects in Language Acquisition through Neural Language Modelscited by 16
  2. 2026Bringing Up a Bilingual BabyLM: Investigating Multilingual Language Acquisition Using Small-Scale Modelscited by 5
  3. 2025Can LLMs Simulate L2-English Dialogue? An Information-Theoretic Analysis of L1-Dependent Biasescited by 1
  4. 2025BLiSS 1.0: Evaluating Bilingual Learner Competence in Second Language Small Language Modelscited by 0
  5. 2026A Study of Crosslinguistic Influence in Language Modelscited by 0
  6. 2026Is Cross-Lingual Transfer in Bilingual Models Human-Like? A Study with Overlapping Word Forms in Dutch and Englishcited by 0
  7. 2026Dango: A Strictly L1-Only Large Language Model for Studying Second Language Acquisitioncited by 0
  8. 2026Beetle: A Bilingual Model Suite for Modelling Second-Language Processingcited by 0