mlsanityData to 14 Oct 2026

PAEFF: Precise Alignment and Enhanced Gated Feature Fusion for Face-Voice Association

arXiv 2505.17002 ↗First posted 22 May 2025cs.CV cs.AIcited by 9

Not on the map. Why?

Abstract (arXiv)

We study the task of learning association between faces and voices, which is gaining interest in the multimodal community lately. These methods suffer from the deliberate crafting of negative mining procedures as well as the reliance on the distant margin parameter. These issues are addressed by learning a joint embedding space in which orthogonality constraints are applied to the fused embeddings of faces and voices. However, embedding spaces of faces and voices possess different characteristics and require spaces to be aligned before fusing them. To this end, we propose a method that accurately aligns the embedding spaces and fuses them with an enhanced gated fusion thereby improving the performance of face-voice association. Extensive experiments on the VoxCeleb dataset reveals the merits of the proposed approach.

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Builds on 20 of its 33 references that are in the map. First posted on arXiv 22 May 2025.

Builds on

20 of 33 references.

References · 20

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2017Multimodal Machine Learning: A Survey and Taxonomycited by 1,013
  3. 2017VoxCeleb: a large-scale speaker identification datasetcited by 732
  4. 2017Gated Multimodal Units for Information Fusioncited by 248
  5. 2019Hyperbolic Image Embeddingscited by 221
  6. 2018Representation Tradeoffs for Hyperbolic Embeddingscited by 206
  7. 2019Multimodal Intelligence: Representation Learning, Information Fusion, and Applicationscited by 89
  8. 2018Seeing Voices and Hearing Faces: Cross-modal biometric matchingcited by 60
  9. 2018Learnable PINs: Cross-Modal Embeddings for Person Identitycited by 60
  10. 2019Utterance-level Aggregation For Speaker Recognition In The Wildcited by 50
  11. 2021Fusion and Orthogonal Projection for Improved Face-Voice Associationcited by 29
  12. 2020Cross-modal Speaker Verification and Recognition: A Multilingual Perspectivecited by 26
8 more
  1. 2021A Multi-View Approach To Audio-Visual Speaker Verificationcited by 23
  2. 2018Disjoint Mapping Network for Cross-modal Matching of Voices and Facescited by 22
  3. 2021Seeking the Shape of Sound: An Adaptive Framework for Learning Voice-Face Associationcited by 21
  4. 2019Deep Latent Space Learning for Cross-modal Mapping of Audio and Visual Signalscited by 20
  5. 2023Single-branch Network for Multimodal Trainingcited by 13
  6. 2023Speaker Recognition in Realistic Scenario Using Multimodal Datacited by 9
  7. 2023DCTM: Dilated Convolutional Transformer Model for Multimodal Engagement Estimation in Conversationcited by 3
  8. 2022Guiding Attention using Partial-Order Relationships for Image Captioningcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 6; 2026: 3.

9 papers.

  1. 2025Face-voice Association in Multilingual Environments (FAME) 2026 Challenge Evaluation Plancited by 6
  2. 2025Linking Faces and Voices Across Languages: Insights from the FAME 2026 Challengecited by 5
  3. 2025RFOP: Rethinking Fusion and Orthogonal Projection for Face-Voice Associationcited by 1
  4. 2025XM-ALIGN: Unified Cross-Modal Embedding Alignment for Face-Voice Associationcited by 1
  5. 2026Face-Voice Association with Inductive Bias for Maximum Class Separationcited by 1
  6. 2026AMR: Adaptive Modality Routing for Multimodal Polyglot Speaker Identificationcited by 1
  7. 2025MuteSwap: Visual-informed Silent Video Identity Conversioncited by 0
  8. 2025Parameter-Efficient Single Collaborative Branch for Recommendationcited by 0
  9. 2026Face-voice Association across LAnguages and Gender (FLAG) 2027 Challenge Evaluation Plancited by 0