# Dynamic Cross Attention for Audio-Visual Person Verification

arXiv: https://arxiv.org/abs/2403.04661 · HTML: https://mlsanity.com/paper/2403.04661/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 37 references that are in the map.
- First posted on arXiv 7 Mar 2024.

## Builds on (most cited first, 18 of 18)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [When Does Label Smoothing Help?](https://mlsanity.com/paper/1906.02629/) (2019)
- [CosFace: Large Margin Cosine Loss for Deep Face Recognition](https://mlsanity.com/paper/1801.09414/) (2018)
- [VoxCeleb: a large-scale speaker identification dataset](https://mlsanity.com/paper/1706.08612/) (2017)
- [ECAPA-TDNN: Emphasized Channel Attention, Propagation and Aggregation in TDNN Based Speaker Verification](https://mlsanity.com/paper/2005.07143/) (2020)
- [Deep Face Recognition: A Survey](https://mlsanity.com/paper/1804.06655/) (2018)
- [Attentive Statistics Pooling for Deep Speaker Embedding](https://mlsanity.com/paper/1803.10963/) (2018)
- [Regularizing Deep Neural Networks by Noise: Its Interpretation and Optimization](https://mlsanity.com/paper/1710.05179/) (2017)
- [M2Lens: Visualizing and Explaining Multimodal Models for Sentiment Analysis](https://mlsanity.com/paper/2107.08264/) (2021)
- [VoxSRC 2020: The Second VoxCeleb Speaker Recognition Challenge](https://mlsanity.com/paper/2012.06867/) (2020)
- [A Multi-View Approach To Audio-Visual Speaker Verification](https://mlsanity.com/paper/2102.06291/) (2021)
- [Noise-tolerant Audio-visual Online Person Verification using an Attention-based Neural Network Fusion](https://mlsanity.com/paper/1811.10813/) (2018)
- [Cross Attentional Audio-Visual Fusion for Dimensional Emotion Recognition](https://mlsanity.com/paper/2111.05222/) (2021)
- [Audio-Visual Fusion for Emotion Recognition in the Valence-Arousal Space Using Joint Cross-Attention](https://mlsanity.com/paper/2209.09068/) (2022)
- [Speaker recognition with two-step multi-modal deep cleansing](https://mlsanity.com/paper/2210.15903/) (2022)
- [Recursive Joint Attention for Audio-Visual Fusion in Regression based Emotion Recognition](https://mlsanity.com/paper/2304.07958/) (2023)
- [Audio-Visual Speaker Verification via Joint Cross-Attention](https://mlsanity.com/paper/2309.16569/) (2023)
- [Cross-modal Audio-visual Co-learning for Text-independent Speaker Verification](https://mlsanity.com/paper/2302.11254/) (2023)

## Built on by (most cited first, 3 of 3)

- [SSAVSV: Towards Unified Model for Self-Supervised Audio-Visual Speaker Verification](https://mlsanity.com/paper/2506.17694/) (2025)
- [SurgAnt-ViVQA: Learning to Anticipate Surgical Events through GRU-Driven Temporal Cross-Attention](https://mlsanity.com/paper/2511.03178/) (2025)
- [Quality-Aware Multimodal Fusion Reveals Implicit Identity in Valence-Arousal Features](https://mlsanity.com/paper/2607.21347/) (2026)

## Abstract (arXiv)

Although person or identity verification has been predominantly explored using individual modalities such as face and voice, audio-visual fusion has recently shown immense potential to outperform unimodal approaches. Audio and visual modalities are often expected to pose strong complementary relationships, which plays a crucial role in effective audio-visual fusion. However, they may not always strongly complement each other, they may also exhibit weak complementary relationships, resulting in poor audio-visual feature representations. In this paper, we propose a Dynamic Cross-Attention (DCA) model that can dynamically select the cross-attended or unattended features on the fly based on the strong or weak complementary relationships, respectively, across audio and visual modalities. In particular, a conditional gating layer is designed to evaluate the contribution of the cross-attention mechanism and choose cross-attended features only when they exhibit strong complementary relationships, otherwise unattended features. Extensive experiments are conducted on the Voxceleb1 dataset to demonstrate the robustness of the proposed model. Results indicate that the proposed model consistently improves the performance on multiple variants of cross-attention while outperforming the state-of-the-art methods.