Interactive In-Meeting Speaker Correction with Human Feedback
Not on the map. Why?
Abstract (arXiv)
Most automatic speech processing systems operate in ``open loop'' mode without user feedback about who said what, yet human-in-the-loop workflows can potentially enable higher accuracy. We propose an LLM-assisted in-meeting speaker correction system that lets users fix speaker attribution errors through brief corrective feedback. After performing streaming ASR and diarization, the system presents concise LLM-generated summaries to help users identify important speaker errors, and it incorporates user feedback by updating the speaker-attributed transcript and adding online speaker enrollments. To make this workflow effective despite errors in speech processing, LLM analysis, and user feedback, we developed several mechanisms to identify the intended correction more precisely. Further, we built an LLM-driven user feedback simulation to evaluate the workflow reprodubilty and at scale. Applied to the AMI headset test set, our system substantially reduces the DER from a streaming baseline (Google ASR + ECAPA) by 31.99% and speaker substitution error by 52.68%. Results of a pilot usability study suggest several avenues to improve the user experience.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 13 of its 28 references that are in the map. First posted on arXiv 22 Sep 2025.
Builds on
13 of 28 references.
References · 13
- 2020ECAPA-TDNN: Emphasized Channel Attention, Propagation and Aggregation in TDNN Based Speaker Verificationcited by 628
- 2023WhisperX: Time-Accurate Speech Transcription of Long-Form Audiocited by 326
- 2023Translating Natural Language to Planning Goals with Large-Language Modelscited by 133
- 2023Generative Speech Recognition Error Correction with Large Language Models and Task-Activating Promptingcited by 72
- 2024A Survey on Speech Large Language Models for Understandingcited by 71
- 2023Whispering LLaMA: A Cross-Modal Generative Error Correction Framework for Speech Recognitioncited by 56
- 2024"We Need Structured Output": Towards User-centered Constraints on Large Language Model Outputcited by 46
- 2024A Taxonomy for Human-LLM Interaction Modes: An Initial Explorationcited by 38
- 2025Just Put a Human in the Loop? Investigating LLM-Assisted Annotation for Subjective Taskscited by 21
- 2021Overlap-aware low-latency online speaker diarization based on end-to-end local segmentationcited by 15
- 2022Absolute decision corrupts absolutely: conservative online speaker diarisationcited by 2
- 2023The Gift of Feedback: Improving ASR Model Quality by Learning from User Corrections through Federated Learningcited by 2
1 more
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Fewer Hallucinations, More Verification: A Three-Stage LLM-Based Framework for ASR Error Correction
- 2026Towards Human-Like Interactive Speech Recognition With Agentic Correction and Semantic Evaluation
- 2026Interactive ASR: Towards Human-Like Interaction and Semantic Coherence Evaluation for Agentic Speech Recognition
- 2026Benchmarking LLM Judges for Voice-Agent Evaluation: Reliability, Calibration, and Human Oversight
- 2023Multi-stage Large Language Model Correction for Speech Recognition
- 2026Evaluating Large Language Models Abilities for Addressee, Turn-change, and Next Speaker Prediction in Meetings
- 2026Rethinking Human-Aligned Evaluation: An Analysis of Semantic Metrics Beyond WER
- 2023Automatically Correcting Large Language Models: Surveying the landscape of diverse self-correction strategies