mlsanityData to 7 Oct 2026

[Call for Papers] The 2nd BabyLM Challenge: Sample-efficient pretraining on a developmentally plausible corpus

arXiv 2404.06214 ↗First posted 9 Apr 2024cs.CLcited by 26

Not on the map. Why?

Abstract (arXiv)

After last year's successful BabyLM Challenge, the competition will be hosted again in 2024/2025. The overarching goals of the challenge remain the same; however, some of the competition rules will be different. The big changes for this year's competition are as follows: First, we replace the loose track with a paper track, which allows (for example) non-model-based submissions, novel cognitively-inspired benchmarks, or analysis techniques. Second, we are relaxing the rules around pretraining data, and will now allow participants to construct their own datasets provided they stay within the 100M-word or 10M-word budget. Third, we introduce a multimodal vision-and-language track, and will release a corpus of 50% text-only and 50% image-text multimodal data as a starting point for LM model training. The purpose of this CfP is to provide rules for this year's challenge, explain these rule changes and their rationale in greater detail, give a timeline of this year's competition, and provide answers to frequently asked questions from last year's challenge.

Key facts. Cited by 26 papers in the map (data through 7 Oct 2026). Builds on 7 of its 16 references that are in the map. First posted on arXiv 9 Apr 2024.

Builds on

7 of 16 references.

References · 7

  1. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  2. 2018Datasheets for Datasetscited by 1,371
  3. 2022GIT: A Generative Image-to-text Transformer for Vision and Languagecited by 536
  4. 2019Connecting Vision and Language with Localized Narrativescited by 225
  5. 2018A standardized Project Gutenberg corpus for statistical analysis of natural language and quantitative linguisticscited by 64
  6. 2023Call for Papers -- The BabyLM Challenge: Sample-efficient pretraining on a developmentally plausible corpuscited by 62
  7. 2023Trained on 100 million words and still in shape: BERT meets British National Corpuscited by 39

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 10; 2025: 14; 2026: 2.

26 papers.

Sort:

By area:

  1. 2024GPT or BERT: why not both?cited by 26
  2. 2025BabyLM Turns 3: Call for papers for the 2025 BabyLM workshopcited by 24
  3. 2025Do Construction Distributions Shape Formal Language Learning In German BabyLMs?cited by 11
  4. 2025Are BabyLMs Deaf to Gricean Maxims? A Pragmatic Evaluation of Sample-efficient Language Modelscited by 11
  5. 2026BabyLM Turns 4 and Goes Multilingual: Call for Papers for the 2026 BabyLM Workshopcited by 10
  6. 2024AntLM: Bridging Causal and Masked Language Modelscited by 8
  7. 2025Subword models struggle with word learning, but surprisal hides itcited by 6
  8. 2025BabyVLM: Data-Efficient Pretraining of VLMs Inspired by Infant Learningcited by 6
  9. 2024BabyLlama-2: Ensemble-Distilled Models Consistently Outperform Teachers With Limited Datacited by 5
  10. 2024Small Language Models Also Work With Small Vocabularies: Probing the Linguistic Abilities of Grapheme- and Phoneme-Based Baby Llamascited by 5
  11. 2024What Should Baby Models Read? Exploring Sample-Efficient Data Composition on Model Performancecited by 4
  12. 2024BabyLM Challenge: Exploring the Effect of Variation Sets on Language Model Training Efficiencycited by 4
14 more
  1. 2025A Distributional Perspective on Word Learning in Neural Language Modelscited by 4
  2. 2025Pretraining Language Models for Diachronic Linguistic Change Discoverycited by 4
  3. 2024Exploring Curriculum Learning for Vision-Language Tasks: A Study on Small-Scale Multimodal Trainingcited by 3
  4. 2025UD-English-CHILDES: A Collected Resource of Gold and Silver Universal Dependencies Trees for Child Language Interactionscited by 3
  5. 2025LongTail-Swap: benchmarking language models' abilities on rare wordscited by 3
  6. 2024BERTtime Stories: Investigating the Role of Synthetic Story Data in Language Pre-trainingcited by 2
  7. 2025Influence-driven Curriculum Learning for Pre-training on Limited Datacited by 2
  8. 2025Model Merging to Maintain Language-Only Performance in Developmentally Plausible Multimodal Modelscited by 1
  9. 2024Choosy Babies Need One Coach: Inducing Mode-Seeking Behavior in BabyLlama with Reverse KL Divergencecited by 0
  10. 2024When Babies Teach Babies: Can student knowledge sharing outperform Teacher-Guided Distillation on small datasets?cited by 0
  11. 2025Syntactic Learnability of Echo State Neural Language Models at Scalecited by 0
  12. 2025Spontaneous Speech Variables for Evaluating LLMs Cognitive Plausibilitycited by 0
  13. 2025PaPaformer: Language Model from Pre-trained Parallel Pathscited by 0
  14. 2026A retrospective analysis on the use of LLMs to study infant syntax learningcited by 0