# Towards Robust Family-Infant Audio Analysis Based on Unsupervised Pretraining of Wav2vec 2.0 on Large-Scale Unlabeled Family Audio

arXiv: https://arxiv.org/abs/2305.12530 · HTML: https://mlsanity.com/paper/2305.12530/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 May 2023.

## Built on by (most cited first, 6 of 6)

- [BabyHuBERT: Multilingual Self-Supervised Learning for Segmenting Speakers in Child-Centered Long-Form Recordings](https://mlsanity.com/paper/2509.15001/) (2025)
- [Exploring Speech Foundation Models for Speaker Diarization in Child-Adult Dyadic Interactions](https://mlsanity.com/paper/2406.07890/) (2024)
- [Automatic Screening for Children with Speech Disorder using Automatic Speech Recognition: Opportunities and Challenges](https://mlsanity.com/paper/2410.11865/) (2024)
- [Mind the Shift: Using Delta SSL Embeddings to Enhance Child ASR](https://mlsanity.com/paper/2601.20142/) (2026)
- [Sound Tagging in Infant-centric Home Soundscapes](https://mlsanity.com/paper/2406.17190/) (2024)
- [Robust Multi-Tier Infant-Centered Audio Understanding with Whisper via Structured Speaker Conditioning](https://mlsanity.com/paper/2608.11587/) (2026)

## Abstract (arXiv)

To perform automatic family audio analysis, past studies have collected recordings using phone, video, or audio-only recording devices like LENA, investigated supervised learning methods, and used or fine-tuned general-purpose embeddings learned from large pretrained models. In this study, we advance the audio component of a new infant wearable multi-modal device called LittleBeats (LB) by learning family audio representation via wav2vec 2.0 (W2V2) pertaining. We show given a limited number of labeled LB home recordings, W2V2 pretrained using 1k-hour of unlabeled home recordings outperforms oracle W2V2 pretrained on 960-hour unlabeled LibriSpeech in terms of parent/infant speaker diarization (SD) and vocalization classifications (VC) at home. Extra relevant external unlabeled and labeled data further benefit W2V2 pretraining and fine-tuning. With SpecAug and environmental speech corruptions, we obtain 12% relative gain on SD and moderate boost on VC. Code and model weights are available.