Discovering Hidden Visual Concepts Beyond Linguistic Input in Infant Learning
Not on the map. Why?
Abstract (arXiv)
Infants develop complex visual understanding rapidly, even preceding the acquisition of linguistic skills. As computer vision seeks to replicate the human vision system, understanding infant visual development may offer valuable insights. In this paper, we present an interdisciplinary study exploring this question: can a computational model that imitates the infant learning process develop broader visual concepts that extend beyond the vocabulary it has heard, similar to how infants naturally learn? To investigate this, we analyze a recently published model in Science by Vong et al., which is trained on longitudinal, egocentric images of a single child paired with transcribed parental speech. We perform neuron labeling to identify visual concept neurons hidden in the model's internal representations. We then demonstrate that these neurons can recognize objects beyond the model's original vocabulary. Furthermore, we compare the differences in representation between infant models and those in modern computer vision models, such as CLIP and ImageNet pre-trained model. Ultimately, our work bridges cognitive science and computer vision by analyzing the internal representations of a computational model trained on an infant visual and linguistic inputs. Project page is available at https://kexueyi.github.io/webpage-discover-hidden-visual-concepts.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 22 of its 47 references that are in the map. First posted on arXiv 9 Jan 2025.
Builds on
22 of 47 references.
References · 22
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
- 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,590
- 2016Aggregated Residual Transformations for Deep Neural Networkscited by 2,540
- 2019Similarity of Neural Network Representations Revisitedcited by 1,695
- 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,530
- 2022Toy Models of Superpositioncited by 928
- 2018Scaling Egocentric Vision: The EPIC-KITCHENS Datasetcited by 789
- 2017Network Dissection: Quantifying Interpretability of Deep Visual Representationscited by 703
- 2023Interpreting CLIP's Image Representation via Text-Based Decompositioncited by 189
- 2020Compositional Explanations of Neuronscited by 172
10 more
- 2022CLIP-Dissect: Automatic Description of Neuron Representations in Deep Vision Networkscited by 144
- 2022Natural Language Descriptions of Deep Visual Featurescited by 136
- 2020Self-supervised learning through the eyes of a childcited by 61
- 2023Rosetta Neurons: Mining the Common Units in a Model Zoocited by 46
- 2023Identifying Interpretable Subspaces in Image Representationscited by 32
- 2023Labeling Neural Representations with Inverse Recognitioncited by 31
- 2020Towards Unique and Informative Captioning of Imagescited by 25
- 2023Learning high-level visual representations from a child's perspective without strong inductive biasescited by 22
- 2024Understanding Visual Feature Reliance through the Lens of Complexitycited by 15
- 2020A Computational Model of Early Word Learning from the Infant's Point of Viewcited by 6
Built on by
Citing papers per year: 2025: 2.
2 papers.