mlsanityData to 14 Oct 2026

Objects Before Words: Object-First Inductive Biases for Grounding Language in Child-View Video

arXiv 2606.12985 ↗First posted 11 Jun 2026cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Learning grounded word meaning from natural experience requires resolving two ambiguities in infant-view recordings: when the named referent appears and where it is in a cluttered frame. In SAYCam-style data, caregiver speech is sparse and weakly synchronized with egocentric video, so single-frame contrastive pairing yields noisy positives in which the intended object is absent or entangled with distractors. We propose BabyMind, an object-first bias for child-view contrastive learning under sparse, noisy supervision. BabyMind extracts candidate object embeddings using an offline mask-based region interface, links candidates across a short utterance-centered window into lightweight object files via tracking, and aligns utterances to bags of object files with a prototype-space multiple-instance contrastive objective. Track-coherence and global-object agreement regularizers stabilize learning and transfer object-file structure into the global frame embedding used at evaluation. On SAYCam-S, BabyMind improves Labeled-S 15 forced-choice accuracy by +2.6 points over CVCL and yields consistent gains on in-vocabulary out-of-distribution benchmarks. Code is available at https://github.com/sathiiii/BabyMind.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 21 of its 34 references that are in the map. First posted on arXiv 11 Jun 2026.

Builds on

21 of 34 references.

References · 21

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  3. 2023Segment Anythingcited by 8,234
  4. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  5. 2017Neural Discrete Representation Learningcited by 4,712
  6. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,857
  7. 2018Unsupervised Feature Learning via Non-Parametric Instance-level Discriminationcited by 1,845
  8. 2019LXMERT: Learning Cross-Modality Encoder Representations from Transformerscited by 1,553
  9. 2018ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustnesscited by 1,432
  10. 2017Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answeringcited by 1,312
  11. 2020Oscar: Object-Semantics Aligned Pre-training for Vision-Language Taskscited by 1,214
  12. 2018Deep Clustering for Unsupervised Learning of Visual Featurescited by 1,028
9 more
  1. 2020Object-Centric Learning with Slot Attentioncited by 876
  2. 2017Time-Contrastive Networks: Self-Supervised Learning from Videocited by 459
  3. 2019MONet: Unsupervised Scene Decomposition and Representationcited by 392
  4. 2021Efficient Visual Pretraining with Contrastive Detectioncited by 150
  5. 2022CLIP-Dissect: Automatic Description of Neuron Representations in Deep Vision Networkscited by 147
  6. 2020Self-supervised learning through the eyes of a childcited by 61
  7. 2026BabyVision: Visual Reasoning Beyond Languagecited by 49
  8. 2023Learning high-level visual representations from a child's perspective without strong inductive biasescited by 22
  9. 2025Perceptual Inductive Bias Is What You Need Before Contrastive Learningcited by 4

Built on by

Not cited yet.