# Egocentric Speaker Classification in Child-Adult Dyadic Interactions: From Sensing to Computational Modeling

arXiv: https://arxiv.org/abs/2409.09340 · HTML: https://mlsanity.com/paper/2409.09340/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 30 references that are in the map.
- First posted on arXiv 14 Sep 2024.

## Builds on (most cited first, 15 of 15)

- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Parameter-Efficient Transfer Learning for NLP](https://mlsanity.com/paper/1902.00751/) (2019)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [Ego4D: Around the World in 3,000 Hours of Egocentric Video](https://mlsanity.com/paper/2110.07058/) (2021)
- [End-to-end speaker segmentation for overlap-aware resegmentation](https://mlsanity.com/paper/2104.04045/) (2021)
- [Benchmarking Children's ASR with Supervised and Self-supervised Speech Foundation Models](https://mlsanity.com/paper/2406.10507/) (2024)
- [Improving child speech recognition with augmented child-like speech](https://mlsanity.com/paper/2406.10284/) (2024)
- [Meta-learning for robust child-adult classification from speech](https://mlsanity.com/paper/1910.11400/) (2019)
- [Enhancing Child Vocalization Classification with Phonetically-Tuned Embeddings for Assisting Autism Diagnosis](https://mlsanity.com/paper/2309.07287/) (2023)
- [Dynamic Speech Endpoint Detection with Regression Targets](https://mlsanity.com/paper/2210.14252/) (2022)
- [Robust Self Supervised Speech Embeddings for Child-Adult Classification in Interactions involving Children with Autism](https://mlsanity.com/paper/2307.16398/) (2023)
- [Understanding Spoken Language Development of Children with ASD Using Pre-trained Speech Embeddings](https://mlsanity.com/paper/2305.14117/) (2023)
- [Data Efficient Child-Adult Speaker Diarization with Simulated Conversations](https://mlsanity.com/paper/2409.08881/) (2024)
- [Audio-visual child-adult speaker classification in dyadic interactions](https://mlsanity.com/paper/2310.01867/) (2023)

## Built on by (most cited first, 1 of 1)

- [Who Said What WSW 2.0? Enhanced Automated Analysis of Preschool Classroom Speech](https://mlsanity.com/paper/2505.09972/) (2025)

## Abstract (arXiv)

Autism spectrum disorder (ASD) is a neurodevelopmental condition characterized by challenges in social communication, repetitive behavior, and sensory processing. One important research area in ASD is evaluating children's behavioral changes over time during treatment. The standard protocol with this objective is BOSCC, which involves dyadic interactions between a child and clinicians performing a pre-defined set of activities. A fundamental aspect of understanding children's behavior in these interactions is automatic speech understanding, particularly identifying who speaks and when. Conventional approaches in this area heavily rely on speech samples recorded from a spectator perspective, and there is limited research on egocentric speech modeling. In this study, we design an experiment to perform speech sampling in BOSCC interviews from an egocentric perspective using wearable sensors and explore pre-training Ego4D speech samples to enhance child-adult speaker classification in dyadic interactions. Our findings highlight the potential of egocentric speech collection and pre-training to improve speaker classification accuracy.