# Multimodal Speaker Identification in Classroom Environments

arXiv: https://arxiv.org/abs/2606.13712 · HTML: https://mlsanity.com/paper/2606.13712/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 27 references that are in the map.
- First posted on arXiv 10 Jun 2026.

## Builds on (most cited first, 12 of 12)

- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Optuna: A Next-generation Hyperparameter Optimization Framework](https://mlsanity.com/paper/1907.10902/) (2019)
- [SpeechBrain: A General-Purpose Speech Toolkit](https://mlsanity.com/paper/2106.04624/) (2021)
- [Measuring Conversational Uptake: A Case Study on Student-Teacher Interactions](https://mlsanity.com/paper/2106.03873/) (2021)
- [Transcribe-to-Diarize: Neural Speaker Diarization for Unlimited Number of Speakers using End-to-End Speaker-Attributed ASR](https://mlsanity.com/paper/2110.03151/) (2021)
- [Deep Speaker Embeddings for Far-Field Speaker Recognition on Short Utterances](https://mlsanity.com/paper/2002.06033/) (2020)
- [The Promises and Pitfalls of Using Language Models to Measure Instruction Quality in Education](https://mlsanity.com/paper/2404.02444/) (2024)
- [LLM-based speaker diarization correction: A generalizable approach](https://mlsanity.com/paper/2406.04927/) (2024)
- [Enhancing Dialogue Annotation with Speaker Characteristics Leveraging a Frozen LLM](https://mlsanity.com/paper/2508.04795/) (2025)
- [Effect of utterance duration and phonetic content on speaker identification using second-order statistical methods](https://mlsanity.com/paper/2402.16429/) (2024)
- [Identifying Speakers in Dialogue Transcripts: A Text-based Approach Using Pretrained Language Models](https://mlsanity.com/paper/2407.12094/) (2024)
- [FT-Boosted SV: Towards Noise Robust Speaker Verification for English Speaking Classroom Environments](https://mlsanity.com/paper/2505.20222/) (2025)

## Abstract (arXiv)

Automated analysis of K-12 classroom dynamics faces challenges due to background noise and variable child speech, often confounding acoustic-only models. This study evaluates a multimodal speaker identification framework anchoring acoustic embeddings with LLM-derived semantic context. Using a subset of the EDSI dataset (8 math classrooms, N = 2,801 utterances), we found an acoustic baseline (ECAPA-TDNN) achieved only 39.0% accuracy. By integrating transcript-based "contextual anchoring" into a gradient boosting classifier, our multimodal approach raised student identification to 50.3%. Performance also improved for utterances over 5 seconds, reaching 76.9% accuracy (vs. 64.9% baseline) with a 90.9% Top-3 accuracy. Additionally, the model distinguished teacher vs. student roles with 99.3% accuracy. This approach advances the feasibility of automated feedback systems capable of considering individual student participation, a crucial step for supporting equitable instruction at scale.