mlsanityData to 14 Oct 2026

The Multimodal Information Based Speech Processing (MISP) 2025 Challenge: Audio-Visual Diarization and Recognition

arXiv 2505.13971 ↗First posted 20 May 2025cs.SD cs.AI eess.AScited by 4

Not on the map. Why?

Abstract (arXiv)

Meetings are a valuable yet challenging scenario for speech applications due to complex acoustic conditions. This paper summarizes the outcomes of the MISP 2025 Challenge, hosted at Interspeech 2025, which focuses on multi-modal, multi-device meeting transcription by incorporating video modality alongside audio. The tasks include Audio-Visual Speaker Diarization (AVSD), Audio-Visual Speech Recognition (AVSR), and Audio-Visual Diarization and Recognition (AVDR). We present the challenge's objectives, tasks, dataset, baseline systems, and solutions proposed by participants. The best-performing systems achieved significant improvements over the baseline: the top AVSD model achieved a Diarization Error Rate (DER) of 8.09%, improving by 7.43%; the top AVSR system achieved a Character Error Rate (CER) of 9.48%, improving by 10.62%; and the best AVDR system achieved a concatenated minimum-permutation Character Error Rate (cpCER) of 11.56%, improving by 72.49%.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 21 of its 30 references that are in the map. First posted on arXiv 20 May 2025.

Builds on

21 of 30 references.

References · 21

  1. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
  2. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,863
  3. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,657
  4. 2018VoxCeleb2: Deep Speaker Recognitioncited by 875
  5. 2020ECAPA-TDNN: Emphasized Channel Attention, Propagation and Aggregation in TDNN Based Speaker Verificationcited by 628
  6. 2021WenetSpeech: A 10000+ Hours Multi-domain Mandarin Corpus for Speech Recognitioncited by 197
  7. 2022Paraformer: Fast and Accurate Parallel Transformer for Non-autoregressive End-to-End Speech Recognitioncited by 144
  8. 2020CHiME-6 Challenge:Tackling Multispeaker Speech Recognition for Unsegmented Recordingscited by 132
  9. 2020Continuous speech separation: dataset and analysiscited by 74
  10. 2021M2MeT: The ICASSP 2022 Multi-Channel Multi-Party Meeting Transcription Challengecited by 71
  11. 2021AISHELL-4: An Open Source Dataset for Speech Enhancement, Separation, Recognition and Speaker Diarization in Conference Scenariocited by 60
  12. 2023MossFormer2: Combining Transformer and RNN-Free Recurrent Network for Enhanced Time-Domain Monaural Speech Separationcited by 39
9 more
  1. 2020Lip-reading with Densely Connected Temporal Convolutional Networkscited by 26
  2. 2024VoxBlink2: A 100K+ Speaker Recognition Corpus and the Open-Set Speaker-Identification Benchmarkcited by 26
  3. 2020DOVER-Lap: A Method for Combining Overlap-aware Diarization Outputscited by 19
  4. 20233D-Speaker: A Large-Scale Multi-Device, Multi-Distance, and Multi-Dialect Corpus for Speech Representation Disentanglementcited by 15
  5. 2022GPU-accelerated Guided Source Separation for Meeting Transcriptioncited by 10
  6. 2023The Multimodal Information based Speech Processing (MISP) 2022 Challenge: Audio-Visual Diarization and Recognitioncited by 5
  7. 2023Improving Audio-Visual Speech Recognition by Lip-Subword Correlation Based Visual Pre-training and Cross-Modal Fusion Encodercited by 3
  8. 2023The Multimodal Information Based Speech Processing (MISP) 2023 Challenge: Audio-Visual Target Speaker Extractioncited by 2
  9. 2024Sequence-to-Sequence Neural Diarization with Automatic Speaker Detection and Representationcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 2.

4 papers.

  1. 2025SpeakerLM: End-to-End Versatile Speaker Diarization and Recognition with Multimodal Large Language Modelscited by 16
  2. 2025M3-SLU: Evaluating Speaker-Attributed Reasoning in Multimodal Large Language Modelscited by 4
  3. 2026OmniTrace: A Unified Framework for Generation-Time Attribution in Omni-Modal LLMscited by 1
  4. 2026MeetingToM: Evaluating Multimodal LLMs on Theory-of-Mind Reasoning in Multi-Party Meetingscited by 1