mlsanityData to 7 Oct 2026

A model of early word acquisition based on realistic-scale audiovisual naming events

arXiv 2406.05259 ↗First posted 7 Jun 2024eess.AS cs.AI cs.CLcited by 3

Not on the map. Why?

Abstract (arXiv)

Infants gradually learn to parse continuous speech into words and connect names with objects, yet the mechanisms behind development of early word perception skills remain unknown. We studied the extent to which early words can be acquired through statistical learning from regularities in audiovisual sensory input. We simulated word learning in infants up to 12 months of age in a realistic setting, using a model that solely learns from statistical regularities in unannotated raw speech and pixel-level visual input. Crucially, the quantity of object naming events was carefully designed to match that accessible to infants of comparable ages. Results show that the model effectively learns to recognize words and associate them with corresponding visual objects, with a vocabulary growth rate comparable to that observed in infants. The findings support the viability of general statistical learning for early word perception, demonstrating how learning can operate without assuming any prior linguistic capabilities.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 26 of its 73 references that are in the map. First posted on arXiv 7 Jun 2024.

Builds on

26 of 73 references.

References · 26

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  4. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  5. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  6. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  7. 2018Jointly Discovering Visual Objects and Spoken Words from Raw Sensory Inputcited by 81
  8. 2016Cognitive Science in the era of Artificial Intelligence: A roadmap for reverse-engineering the infant language-learnercited by 68
  9. 2020Text-Free Image-to-Speech Synthesis Using Learned Segmental Unitscited by 50
  10. 2019Learning Hierarchical Discrete Linguistic Units from Visually-Grounded Speechcited by 44
  11. 2017Representations of language in a model of visually grounded speech signalcited by 37
  12. 2022Word Discovery in Visually Grounded, Self-Supervised Speech Modelscited by 37
14 more
  1. 2022Self-supervised language learning from raw audio: Lessons from the Zero Resource Speech Challengecited by 34
  2. 2019Large-scale representation learning from visually grounded untranscribed speechcited by 33
  3. 2021Visually grounded models of spoken language: A survey of datasets, architectures and evaluation techniquescited by 32
  4. 2023BabySLM: language-acquisition-friendly benchmark of self-supervised spoken language modelscited by 29
  5. 2017Encoding of phonology in a recurrent neural model of grounded speechcited by 27
  6. 2022Self-Supervised Representation Learning for Speech Using Visual Grounding and Masked Language Modelingcited by 25
  7. 2021Fast-Slow Transformer for Visually Grounding Speechcited by 23
  8. 2019Word Recognition, Competition, and Activation in a Model of Visually Grounded Speechcited by 13
  9. 2021Can phones, syllables, and words emerge as side-products of cross-situational audiovisual learning? -- A computational investigationcited by 12
  10. 2023Syllable Discovery and Cross-Lingual Generalization in a Visually Grounded, Self-Supervised Speech Modelcited by 11
  11. 2019A computational model of early language acquisition from audiovisual experiences of young infantscited by 10
  12. 2021Evaluation of Audio-Visual Alignments in Visually Grounded Speech Modelscited by 8
  13. 2022Learning English with Peppa Pigcited by 8
  14. 2023Simultaneous or Sequential Training? How Speech Representations Cooperate in a Multi-Task Self-Supervised Learning Systemcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 2.

3 papers.

  1. 2026Computational modeling of early language learning from acoustic speech and audiovisual input without linguistic priorscited by 1
  2. 2025Do Blind Spots Matter for Word-Referent Mapping? A Computational Study with Infant Egocentric Videocited by 0
  3. 2026EgoBabyVLM: Benchmarking Cross-Modal Learning from Naturalistic Egocentric Video Datacited by 0