mlsanityData to 14 Oct 2026

Learning Speaker Identity Beyond Language and Modality Constraints: Insights from the POLY-SIM 2026 Challenge

arXiv 2607.13669 ↗First posted 15 Jul 2026cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Multimodal speaker identification systems typically assume the availability of complete and homogeneous audio-visual modalities during both training and testing, and assume each speaker only speaks a single language. However, in real-world applications, such assumptions often do not hold. Visual or audio information may be missing due to occlusions, camera or microphone failures, or privacy constraints. Multilingual speakers introduce additional complexity due to linguistic variability across languages. These situations constitute substantial challenges for the robustness and generalization capabilities of multimodal speaker identification systems. Aim of the POLY-SIM 2026 challenge is to address these aspects of speaker identification and to provide a standardized setup for the comparison of the proposed solutions.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 21 of its 33 references that are in the map. First posted on arXiv 15 Jul 2026.

Builds on

21 of 33 references.

References · 21

  1. 2015FaceNet: A Unified Embedding for Face Recognition and Clusteringcited by 2,849
  2. 2018VoxCeleb2: Deep Speaker Recognitioncited by 875
  3. 2017VoxCeleb: a large-scale speaker identification datasetcited by 732
  4. 2020ECAPA-TDNN: Emphasized Channel Attention, Propagation and Aggregation in TDNN Based Speaker Verificationcited by 628
  5. 2022Are Multimodal Transformers Robust to Missing Modality?cited by 186
  6. 2021Is Someone Speaking? Exploring Long-term Temporal Features for Audio-visual Active Speaker Detectioncited by 113
  7. 2021Fusion and Orthogonal Projection for Improved Face-Voice Associationcited by 29
  8. 2020Cross-modal Speaker Verification and Recognition: A Multilingual Perspectivecited by 26
  9. 2024VoxBlink2: A 100K+ Speaker Recognition Corpus and the Open-Set Speaker-Identification Benchmarkcited by 26
  10. 2020Audio-visual Speaker Recognition with a Cross-modal Discriminative Networkcited by 24
  11. 2024A Multimodal Single-Branch Embedding Network for Recommendation in Cold-Start and Missing Modality Scenarioscited by 13
  12. 2023Target Active Speaker Detection with Audio-visual Cuescited by 11
9 more
  1. 2025Face-voice Association in Multilingual Environments (FAME) 2026 Challenge Evaluation Plancited by 6
  2. 2025Linking Faces and Voices Across Languages: Insights from the FAME 2026 Challengecited by 5
  3. 2024Modality Invariant Multimodal Learning to Handle Missing Modalities: A Single-Branch Approachcited by 4
  4. 2025Single-Branch Network Architectures to Close the Modality Gap in Multimodal Recommendationcited by 3
  5. 2026Robust Harmful Meme Detection under Missing Modalities via Shared Representation Learningcited by 2
  6. 2026POLY-SIM: Polyglot Speaker Identification with Missing Modality Grand Challenge 2026 Evaluation Plancited by 1
  7. 2026Missing-Token Prompted Reliability-Aware Fusion for Robust Polyglot Speaker Identificationcited by 1
  8. 2026MaskedFOP: Polyglot Speaker Identification under Missing Visual Modality via Cascaded Graph Label Propagationcited by 1
  9. 2026AMR: Adaptive Modality Routing for Multimodal Polyglot Speaker Identificationcited by 1

Built on by

Not cited yet.