# ChildPlay: A New Benchmark for Understanding Children's Gaze Behaviour

arXiv: https://arxiv.org/abs/2307.01630 · HTML: https://mlsanity.com/paper/2307.01630/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 69 references that are in the map.
- First posted on arXiv 4 Jul 2023.

## Builds on (most cited first, 24 of 24)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [Masked-attention Mask Transformer for Universal Image Segmentation](https://mlsanity.com/paper/2112.01527/) (2021)
- [Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfer](https://mlsanity.com/paper/1907.01341/) (2019)
- [MegaDepth: Learning Single-View Depth Prediction from Internet Photos](https://mlsanity.com/paper/1804.00607/) (2018)
- [AVA: A Video Dataset of Spatio-temporally Localized Atomic Visual Actions](https://mlsanity.com/paper/1705.08421/) (2017)
- [Learning to Recover 3D Scene Shape from a Single Image](https://mlsanity.com/paper/2012.09365/) (2020)
- [Gaze360: Physically Unconstrained Gaze Estimation in the Wild](https://mlsanity.com/paper/1910.10088/) (2019)
- [Detecting Attended Visual Targets in Video](https://mlsanity.com/paper/2003.02501/) (2020)
- [DiverseDepth: Affine-invariant Depth Prediction Using Diverse Data](https://mlsanity.com/paper/2002.00569/) (2020)
- [Connecting Gaze, Scene, and Attention: Generalized Attention Estimation via Joint Modeling of Gaze and Scene Saliency](https://mlsanity.com/paper/1807.10437/) (2018)
- [End-to-End Human-Gaze-Target Detection with Transformers](https://mlsanity.com/paper/2203.10433/) (2022)
- [Understanding Human Gaze Communication by Spatio-Temporal Graph Reasoning](https://mlsanity.com/paper/1909.02144/) (2019)
- [Believe It or Not, We Know What You Are Looking at!](https://mlsanity.com/paper/1907.02364/) (2019)
- [LAEO-Net: revisiting people Looking At Each Other in videos](https://mlsanity.com/paper/1906.05261/) (2019)
- [Personalized Machine Learning for Robot Perception of Affect and Engagement in Autism Therapy](https://mlsanity.com/paper/1802.01186/) (2018)
- [A Differential Approach for Gaze Estimation](https://mlsanity.com/paper/1904.09459/) (2019)
- [GOO: A Dataset for Gaze Object Prediction in Retail Environments](https://mlsanity.com/paper/2105.10793/) (2021)
- [GaTector: A Unified Framework for Gaze Object Prediction](https://mlsanity.com/paper/2112.03549/) (2021)
- [Detecting Gaze Towards Eyes in Natural Social Interactions and its Use in Child Assessment](https://mlsanity.com/paper/1902.00607/) (2019)
- [Single-Stage 3D Geometry-Preserving Depth Estimation Model Training on Dataset Mixtures with Uncalibrated Stereo Data](https://mlsanity.com/paper/2306.02878/) (2023)
- [Do Pedestrians Pay Attention? Eye Contact Detection in the Wild](https://mlsanity.com/paper/2112.04212/) (2021)

## Built on by (most cited first, 30 of 35)

- [Gaze-LLE: Gaze Target Estimation via Large-Scale Learned Encoders](https://mlsanity.com/paper/2412.09586/) (2024)
- [Learning from Observer Gaze:Zero-Shot Attention Prediction Oriented by Human-Object Interaction Recognition](https://mlsanity.com/paper/2405.09931/) (2024)
- [Exploring the Zero-Shot Capabilities of Vision-Language Models for Improving Gaze Following](https://mlsanity.com/paper/2406.03907/) (2024)
- [Towards Online Multi-Modal Social Interaction Understanding](https://mlsanity.com/paper/2503.19851/) (2025)
- [Enhancing 3D Gaze Estimation in the Wild using Weak Supervision with Gaze Following Labels](https://mlsanity.com/paper/2502.20249/) (2025)
- [GazeVLM: A Vision-Language Model for Multi-Task Gaze Understanding](https://mlsanity.com/paper/2511.06348/) (2025)
- [GazeHTA: End-to-end Gaze Target Detection with Head-Target Association](https://mlsanity.com/paper/2404.10718/) (2024)
- [Diffusion-Refined VQA Annotations for Semi-Supervised Gaze Following](https://mlsanity.com/paper/2406.02774/) (2024)
- [Multi-view Gaze Target Estimation](https://mlsanity.com/paper/2508.05857/) (2025)
- [Omni-MMSI: Toward Identity-attributed Social Interaction Understanding](https://mlsanity.com/paper/2604.00267/) (2026)
- [A Novel Framework for Multi-Person Temporal Gaze Following and Social Gaze Prediction](https://mlsanity.com/paper/2403.10511/) (2024)
- [GA3CE: Unconstrained 3D Gaze Estimation with Gaze-Aware 3D Context Encoding](https://mlsanity.com/paper/2505.10671/) (2025)
- [Learning in Focus: Detecting Behavioral and Collaborative Engagement Using Vision Transformers](https://mlsanity.com/paper/2508.15782/) (2025)
- [VL4Gaze: Unleashing Vision-Language Models for Gaze Following](https://mlsanity.com/paper/2512.20735/) (2025)
- [ChildPlay-Hand: A Dataset of Hand Manipulations in the Wild](https://mlsanity.com/paper/2409.09319/) (2024)
- [MOOSE: Pay Attention to Temporal Dynamics for Video Understanding via Optical Flows](https://mlsanity.com/paper/2506.01119/) (2025)
- [GazeDETR: Gaze Detection using Disentangled Head and Gaze Representations](https://mlsanity.com/paper/2508.12966/) (2025)
- [Age-Inclusive 3D Human Mesh Recovery for Action-Preserving Data Anonymization](https://mlsanity.com/paper/2512.05259/) (2025)
- [Decoding Children's Gait Behavior](https://mlsanity.com/paper/2608.00371/) (2026)
- [CAPE: A CLIP-Aware Pointing Ensemble of Complementary Heatmap Cues for Embodied Reference Understanding](https://mlsanity.com/paper/2507.21888/) (2025)
- [CSGaze: Context-aware Social Gaze Prediction](https://mlsanity.com/paper/2511.05955/) (2025)
- [GazeMoE: Perception of Gaze Target with Mixture-of-Experts](https://mlsanity.com/paper/2603.06256/) (2026)
- [End-to-End Shared Attention Estimation via Group Detection with Feedback Refinement](https://mlsanity.com/paper/2604.01714/) (2026)
- [Eyes on VLM: Benchmarking Gaze Following and Social Gaze Prediction in Vision Language Models](https://mlsanity.com/paper/2605.19859/) (2026)
- [OmniGF: A Dual-Branch Vision-Language Framework for Unified Gaze Following](https://mlsanity.com/paper/2605.26399/) (2026)
- [Multi-scale Object-Aware Gaze Estimation via Geometric Reasoning](https://mlsanity.com/paper/2606.29334/) (2026)
- [SAGE: Synchronized Action-Gaze Recognition and Anticipation for Human Behavior Understanding](https://mlsanity.com/paper/2607.04017/) (2026)
- [PAGE: Towards Practical Human-level Gaze Target Estimation](https://mlsanity.com/paper/2607.04860/) (2026)
- [TextGaze: Prompting Gaze Target Estimation with Textual Scene Cues](https://mlsanity.com/paper/2607.10130/) (2026)
- [Open-Vocabulary Gaze Object Prediction: Benchmark and Method](https://mlsanity.com/paper/2607.18827/) (2026)

## Abstract (arXiv)

Gaze behaviors such as eye-contact or shared attention are important markers for diagnosing developmental disorders in children. While previous studies have looked at some of these elements, the analysis is usually performed on private datasets and is restricted to lab settings. Furthermore, all publicly available gaze target prediction benchmarks mostly contain instances of adults, which makes models trained on them less applicable to scenarios with young children. In this paper, we propose the first study for predicting the gaze target of children and interacting adults. To this end, we introduce the ChildPlay dataset: a curated collection of short video clips featuring children playing and interacting with adults in uncontrolled environments (e.g. kindergarten, therapy centers, preschools etc.), which we annotate with rich gaze information. We further propose a new model for gaze target prediction that is geometrically grounded by explicitly identifying the scene parts in the 3D field of view (3DFoV) of the person, leveraging recent geometry preserving depth inference methods. Our model achieves state of the art results on benchmark datasets and ChildPlay. Furthermore, results show that looking at faces prediction performance on children is much worse than on adults, and can be significantly improved by fine-tuning models using child gaze annotations. Our dataset and models will be made publicly available.