# Cross-modal Speaker Verification and Recognition: A Multilingual Perspective

arXiv: https://arxiv.org/abs/2004.13780 · HTML: https://mlsanity.com/paper/2004.13780/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Apr 2020.

## Built on by (most cited first, 26 of 26)

- [Fusion and Orthogonal Projection for Improved Face-Voice Association](https://mlsanity.com/paper/2112.10483/) (2021)
- [Single-branch Network for Multimodal Training](https://mlsanity.com/paper/2303.06129/) (2023)
- [Speaker Recognition in Realistic Scenario Using Multimodal Data](https://mlsanity.com/paper/2302.13033/) (2023)
- [PAEFF: Precise Alignment and Enhanced Gated Feature Fusion for Face-Voice Association](https://mlsanity.com/paper/2505.17002/) (2025)
- [Cross-Modal Perceptionist: Can Face Geometry be Gleaned from Voices?](https://mlsanity.com/paper/2203.09824/) (2022)
- [Unsupervised Voice-Face Representation Learning by Cross-Modal Prototype Contrast](https://mlsanity.com/paper/2204.14057/) (2022)
- [Face-voice Association in Multilingual Environments (FAME) 2026 Challenge Evaluation Plan](https://mlsanity.com/paper/2508.04592/) (2025)
- [Linking Faces and Voices Across Languages: Insights from the FAME 2026 Challenge](https://mlsanity.com/paper/2512.20376/) (2025)
- [Audio-Visual Speaker Verification via Joint Cross-Attention](https://mlsanity.com/paper/2309.16569/) (2023)
- [Face-voice Association in Multilingual Environments (FAME) Challenge 2024 Evaluation Plan](https://mlsanity.com/paper/2404.09342/) (2024)
- [FaceSpeak: Expressive and High-Quality Speech Synthesis from Human Portraits of Different Styles](https://mlsanity.com/paper/2501.03181/) (2025)
- [Learning Branched Fusion and Orthogonal Projection for Face-Voice Association](https://mlsanity.com/paper/2208.10238/) (2022)
- [Audio Representation Learning by Distilling Video as Privileged Information](https://mlsanity.com/paper/2302.02845/) (2023)
- [Rethinking Voice-Face Correlation: A Geometry View](https://mlsanity.com/paper/2307.13948/) (2023)
- [Listen Then See: Video Alignment with Speaker Attention](https://mlsanity.com/paper/2404.13530/) (2024)
- [Exploring Robust Face-Voice Matching in Multilingual Environments](https://mlsanity.com/paper/2407.19875/) (2024)
- [Contrastive Learning-based Chaining-Cluster for Multilingual Voice-Face Association](https://mlsanity.com/paper/2408.02025/) (2024)
- [RFOP: Rethinking Fusion and Orthogonal Projection for Face-Voice Association](https://mlsanity.com/paper/2512.02860/) (2025)
- [XM-ALIGN: Unified Cross-Modal Embedding Alignment for Face-Voice Association](https://mlsanity.com/paper/2512.06757/) (2025)
- [Face-Voice Association with Inductive Bias for Maximum Class Separation](https://mlsanity.com/paper/2601.13651/) (2026)
- [POLY-SIM: Polyglot Speaker Identification with Missing Modality Grand Challenge 2026 Evaluation Plan](https://mlsanity.com/paper/2603.24569/) (2026)
- [AMR: Adaptive Modality Routing for Multimodal Polyglot Speaker Identification](https://mlsanity.com/paper/2606.29335/) (2026)
- [Disentangling Speaker and Language Effects in Cross-Lingual Speaker Verification for Iberian Languages](https://mlsanity.com/paper/2607.01161/) (2026)
- [Learning Speaker Identity Beyond Language and Modality Constraints: Insights from the POLY-SIM 2026 Challenge](https://mlsanity.com/paper/2607.13669/) (2026)
- [Face and Voice Cross-modal Association with Learning Convex Feature Embedding](https://mlsanity.com/paper/2607.28129/) (2026)
- [Face-voice Association across LAnguages and Gender (FLAG) 2027 Challenge Evaluation Plan](https://mlsanity.com/paper/2609.17913/) (2026)

## Abstract (arXiv)

Recent years have seen a surge in finding association between faces and voices within a cross-modal biometric application along with speaker recognition. Inspired from this, we introduce a challenging task in establishing association between faces and voices across multiple languages spoken by the same set of persons. The aim of this paper is to answer two closely related questions: "Is face-voice association language independent?" and "Can a speaker be recognised irrespective of the spoken language?". These two questions are very important to understand effectiveness and to boost development of multilingual biometric systems. To answer them, we collected a Multilingual Audio-Visual dataset, containing human speech clips of 154 identities with 3 language annotations extracted from various videos uploaded online. Extensive experiments on the three splits of the proposed dataset have been performed to investigate and answer these novel research questions that clearly point out the relevance of the multilingual problem.