# The BabyView dataset: High-resolution egocentric videos of infants' and young children's everyday experiences

arXiv: https://arxiv.org/abs/2406.10447 · HTML: https://mlsanity.com/paper/2406.10447/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 69 references that are in the map.
- First posted on arXiv 14 Jun 2024.

## Builds on (most cited first, 30 of 31)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [Ego4D: Around the World in 3,000 Hours of Egocentric Video](https://mlsanity.com/paper/2110.07058/) (2021)
- [Deep High-Resolution Representation Learning for Human Pose Estimation](https://mlsanity.com/paper/1902.09212/) (2019)
- [COCO-Stuff: Thing and Stuff Classes in Context](https://mlsanity.com/paper/1612.03716/) (2016)
- [Scaling Egocentric Vision: The EPIC-KITCHENS Dataset](https://mlsanity.com/paper/1804.02748/) (2018)
- [BLiMP: The Benchmark of Linguistic Minimal Pairs for English](https://mlsanity.com/paper/1912.00582/) (2019)
- [Rescaling Egocentric Vision](https://mlsanity.com/paper/2006.13256/) (2020)
- [ViTPose: Simple Vision Transformer Baselines for Human Pose Estimation](https://mlsanity.com/paper/2204.12484/) (2022)
- [RTMPose: Real-Time Multi-Person Pose Estimation based on MMPose](https://mlsanity.com/paper/2303.07399/) (2023)
- [Distil-Whisper: Robust Knowledge Distillation via Large-Scale Pseudo Labelling](https://mlsanity.com/paper/2311.00430/) (2023)
- [SimCC: a Simple Coordinate Classification Perspective for Human Pose Estimation](https://mlsanity.com/paper/2107.03332/) (2021)
- [YOLO-Pose: Enhancing YOLO for Multi Person Pose Estimation Using Object Keypoint Similarity Loss](https://mlsanity.com/paper/2204.06806/) (2022)
- [Self-supervised learning through the eyes of a child](https://mlsanity.com/paper/2007.16189/) (2020)
- [RTMO: Towards High-Performance One-Stage Real-Time Multi-Person Pose Estimation](https://mlsanity.com/paper/2312.07526/) (2023)
- [Unsupervised Learning from Video with Deep Neural Embeddings](https://mlsanity.com/paper/1905.11954/) (2019)
- [Learning high-level visual representations from a child's perspective without strong inductive biases](https://mlsanity.com/paper/2305.15372/) (2023)
- [Visual Grounding Helps Learn Word Meanings in Low-Data Regimes](https://mlsanity.com/paper/2310.13257/) (2023)
- [An open-source voice type classifier for child-centered daylong recordings](https://mlsanity.com/paper/2005.12656/) (2020)
- [High Definition, Inexpensive, Underwater Mapping](https://mlsanity.com/paper/2203.05640/) (2022)
- [The developmental trajectory of object recognition robustness: children are like small adults but unlike big deep neural networks](https://mlsanity.com/paper/2205.10144/) (2022)
- [A systematic investigation of learnability from single child linguistic input](https://mlsanity.com/paper/2402.07899/) (2024)
- [Lexicon-Level Contrastive Visual-Grounding Improves Language Modeling](https://mlsanity.com/paper/2403.14551/) (2024)

## Built on by (most cited first, 17 of 17)

- [Precise Action-to-Video Generation Through Visual Action Prompts](https://mlsanity.com/paper/2508.13104/) (2025)
- [BabyVLM: Data-Efficient Pretraining of VLMs Inspired by Infant Learning](https://mlsanity.com/paper/2504.09426/) (2025)
- [On the robustness of modeling grounded word learning through a child's egocentric input](https://mlsanity.com/paper/2507.14749/) (2025)
- [BabyVLM-V2: Toward Developmentally Grounded Pretraining and Benchmarking of Vision Foundation Models](https://mlsanity.com/paper/2512.10932/) (2025)
- [Challenges in Automated Processing of Speech from Child Wearables: The Case of Voice Type Classifier](https://mlsanity.com/paper/2506.11074/) (2025)
- [Assessing the alignment between infants' visual and linguistic experience using multimodal language models](https://mlsanity.com/paper/2511.18824/) (2025)
- [Human-level 3D shape perception emerges from multi-view learning](https://mlsanity.com/paper/2602.17650/) (2026)
- [Baby Scale: Investigating Models Trained on Individual Children's Language Input](https://mlsanity.com/paper/2603.29522/) (2026)
- [Zero-shot World Models Are Developmentally Efficient Learners](https://mlsanity.com/paper/2604.10333/) (2026)
- [HalluWorld: A Controlled Benchmark for Hallucination via Reference World Models](https://mlsanity.com/paper/2605.19341/) (2026)
- [Human Gaze Boosts Object-Centered Representation Learning](https://mlsanity.com/paper/2501.02966/) (2025)
- [Temporal Slowness in Central Vision Drives Semantic Object Learning](https://mlsanity.com/paper/2602.04462/) (2026)
- [Characterizing the visual representation of objects from the child's view](https://mlsanity.com/paper/2605.14990/) (2026)
- [EgoBabyVLM: Benchmarking Cross-Modal Learning from Naturalistic Egocentric Video Data](https://mlsanity.com/paper/2605.19130/) (2026)
- [Making Sense of Touch from the Child's View for Contrastive Learning](https://mlsanity.com/paper/2606.31943/) (2026)
- [Children, but not language models, show accelerating returns in word learning](https://mlsanity.com/paper/2608.17120/) (2026)
- [More accurate behavioral predictions with hybrid Bayesian-connectionist models](https://mlsanity.com/paper/2608.22154/) (2026)

## Abstract (arXiv)

Human children far exceed modern machine learning algorithms in their sample efficiency, achieving high performance in key domains with much less data than current models. This ''data gap'' is a key challenge both for building intelligent artificial systems and for understanding human development. Egocentric video capturing children's experience--their ''training data''--is a key ingredient for comparison of humans and models and for the development of algorithmic innovations to bridge this gap. Yet there are few such datasets available, and extant data are low-resolution, have limited metadata, and importantly, represent only a small set of children's experiences. Here, we provide the first release of a large developmental egocentric video dataset--the BabyView dataset--recorded using a high-resolution camera with a large vertical field-of-view and gyroscope/accelerometer data. This 868 hour dataset includes egocentric videos from children spanning 6 months to 3 years of age in longitudinal, at-home contexts. We provide gold-standard annotations for the evaluation of speech transcription, speaker diarization, and human pose estimation, and evaluate models in each of these domains. We train self-supervised language and vision models and evaluate their transfer to out-of-distribution tasks, including syntactic structure learning, object recognition, depth estimation, and image segmentation. Although performance in each domain scales with dataset size, overall performance is relatively lower than when models are trained on curated datasets, especially in the visual domain. Our dataset stands as an open challenge for robust, human-like AI systems: how can such systems achieve human-levels of success on the same scale and distribution of training data as humans?