mlsanityData to 19 Oct 2026

The FlySpeech Audio-Visual Speaker Diarization System for MISP Challenge 2022

arXiv 2307.15400 ↗First posted 28 Jul 2023cs.SD eess.AScited by 2

On the map: Speech and audio language models → Speaker recognition and diarization

Abstract (arXiv)

This paper describes the FlySpeech speaker diarization system submitted to the second Multimodal Information Based Speech Processing~(MISP) Challenge held in ICASSP 2022. We develop an end-to-end audio-visual speaker diarization~(AVSD) system, which consists of a lip encoder, a speaker encoder, and an audio-visual decoder. Specifically, to mitigate the degradation of diarization performance caused by separate training, we jointly train the speaker encoder and the audio-visual decoder. In addition, we leverage the large-data pretrained speaker extractor to initialize the speaker encoder.

Key facts. Cited by 2 papers in the map (data through 19 Oct 2026). Placed in Speaker recognition and diarization, one of 11 directions in Speech and audio language models. Builds on 7 of its 11 references that are in the map. First posted on arXiv 28 Jul 2023.

Builds on

7 of 11 references.

Closest ancestors · 1

Same direction, newest first

  1. 2022MFA-Conformer: Multi-scale Feature Aggregation Conformer for Automatic Speaker Verification29 Mar

Other references · 6

  1. 2020ECAPA-TDNN: Emphasized Channel Attention, Propagation and Aggregation in TDNN Based Speaker Verificationcited by 980
  2. 2020CHiME-6 Challenge:Tackling Multispeaker Speech Recognition for Unsegmented Recordingscited by 231
  3. 2020Target-Speaker Voice Activity Detection: a Novel Approach for Multi-Speaker Diarization in a Dinner Party Scenariocited by 133
  4. 2021Is Someone Speaking? Exploring Long-term Temporal Features for Audio-visual Active Speaker Detectioncited by 125
  5. 2020Clova Baseline System for the VoxCeleb Speaker Recognition Challenge 2020cited by 74
  6. 2020S-vectors and TESA: Speaker Embeddings and a Speaker Authenticator Based on Transformer Encodercited by 13

Built on by

Cited by year

Citing papers per year: 2024: 1; 2025: 1.

2 papers.

  1. 2024Multi-Input Multi-Output Target-Speaker Voice Activity Detection For Unified, Flexible, and Robust Audio-Visual Speaker Diarizationcited by 9
  2. 2025Overlap-Adaptive Hybrid Speaker Diarization and ASR-Aware Observation Addition for MISP 2025 Challengecited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2023Joint Training or Not: An Exploration of Pre-trained Speech Models in Audio-Visual Speaker Diarization
  2. 2022Late Audio-Visual Fusion for In-The-Wild Speaker Diarization
  3. 2025The Multimodal Information Based Speech Processing (MISP) 2025 Challenge: Audio-Visual Diarization and Recognition
  4. 2021AVA-AVD: Audio-Visual Speaker Diarization in the Wild
  5. 2023The NPU-ASLP System for Audio-Visual Speech Recognition in MISP 2022 Challenge
  6. 2024Audio-Visual Speaker Diarization: Current Databases, Approaches and Challenges
  7. 2023The Multimodal Information Based Speech Processing (MISP) 2023 Challenge: Audio-Visual Target Speaker Extraction
  8. 2024Quality-Aware End-to-End Audio-Visual Neural Speaker Diarization