# BabyHuBERT: Multilingual Self-Supervised Learning for Segmenting Speakers in Child-Centered Long-Form Recordings

arXiv: https://arxiv.org/abs/2509.15001 · HTML: https://mlsanity.com/paper/2509.15001/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 45 references that are in the map.
- First posted on arXiv 18 Sep 2025.

## Builds on (most cited first, 15 of 15)

- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units](https://mlsanity.com/paper/2106.07447/) (2021)
- [WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing](https://mlsanity.com/paper/2110.13900/) (2021)
- [SUPERB: Speech processing Universal PERformance Benchmark](https://mlsanity.com/paper/2105.01051/) (2021)
- [Speaker Recognition from Raw Waveform with SincNet](https://mlsanity.com/paper/1808.00158/) (2018)
- [Systematic Inequalities in Language Technology Performance across the World's Languages](https://mlsanity.com/paper/2110.06733/) (2021)
- [mHuBERT-147: A Compact Multilingual HuBERT Model](https://mlsanity.com/paper/2406.06371/) (2024)
- [TorchAudio 2.1: Advancing speech recognition, self-supervised learning, and audio processing components for PyTorch](https://mlsanity.com/paper/2310.17864/) (2023)
- [An open-source voice type classifier for child-centered daylong recordings](https://mlsanity.com/paper/2005.12656/) (2020)
- [Speaker detection in the wild: Lessons learned from JSALT 2019](https://mlsanity.com/paper/1912.00938/) (2019)
- [Towards Robust Family-Infant Audio Analysis Based on Unsupervised Pretraining of Wav2vec 2.0 on Large-Scale Unlabeled Family Audio](https://mlsanity.com/paper/2305.12530/) (2023)
- [Self-Supervised Models for Phoneme Recognition: Applications in Children's Speech for Reading Learning](https://mlsanity.com/paper/2503.04710/) (2025)
- [Fifteen Years of Child-Centered Long-Form Recordings: Promises, Resources, and Remaining Challenges to Validity](https://mlsanity.com/paper/2506.11075/) (2025)
- [Challenges in Automated Processing of Speech from Child Wearables: The Case of Voice Type Classifier](https://mlsanity.com/paper/2506.11074/) (2025)
- [Employing self-supervised learning models for cross-linguistic child speech maturity classification](https://mlsanity.com/paper/2506.08999/) (2025)

## Built on by (most cited first, 5 of 5)

- [EgoBabyVLM: Benchmarking Cross-Modal Learning from Naturalistic Egocentric Video Data](https://mlsanity.com/paper/2605.19130/) (2026)
- [Context-aware child-directed speech detection from long-form recordings](https://mlsanity.com/paper/2606.01134/) (2026)
- [Deriving Benchmarking Datasets from Long-Form Recordings: Challenges and Opportunities](https://mlsanity.com/paper/2607.03201/) (2026)
- [Self-Supervised Speech Representations Track Spoken Language Convergence to Adult Models in Infants and Children Who Are Deaf/Hard-of-Hearing](https://mlsanity.com/paper/2608.20396/) (2026)
- [Cross-Linguistic Effects in Bilingual Phoneme BabyLMs](https://mlsanity.com/paper/2609.37121/) (2026)

## Abstract (arXiv)

Child-centered daylong recordings are essential for studying early language development, but existing speech models trained on clean adult data perform poorly due to acoustic and linguistic differences. We introduce BabyHuBERT, a self-supervised speech model trained on 13,000 hours of multilingual child-centered recordings from 40+ languages. Evaluated on voice type classification, the task of identifying who produces speech and when in child-centered recordings (key child, other children, male, and female adults), BabyHuBERT-VTC achieves F1-scores from 55.0% to 76.1% across six corpora, consistently outperforming W2V2-LL4300 and HuBERT (pretrained on English daylongs and clean adult speech, respectively). Notable gains include 14.0 and 18.3 absolute F1 points over HuBERT on Vanuatu and Solomon Islands, demonstrating effectiveness on underrepresented languages. We share code and models to support researchers working with child-centered recordings across diverse linguistic contexts.