# Quality-Aware Multimodal Biometric Recognition

arXiv: https://arxiv.org/abs/2112.05827 · HTML: https://mlsanity.com/paper/2112.05827/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 106 references that are in the map.
- First posted on arXiv 10 Dec 2021.

## Builds on (most cited first, 30 of 38)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [Group Normalization](https://mlsanity.com/paper/1803.08494/) (2018)
- [Show, Attend and Tell: Neural Image Caption Generation with Visual Attention](https://mlsanity.com/paper/1502.03044/) (2015)
- [CosFace: Large Margin Cosine Loss for Deep Face Recognition](https://mlsanity.com/paper/1801.09414/) (2018)
- [VGGFace2: A dataset for recognising faces across pose and age](https://mlsanity.com/paper/1710.08092/) (2017)
- [Joint Face Detection and Alignment using Multi-task Cascaded Convolutional Networks](https://mlsanity.com/paper/1604.02878/) (2016)
- [SphereFace: Deep Hypersphere Embedding for Face Recognition](https://mlsanity.com/paper/1704.08063/) (2017)
- [Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networks](https://mlsanity.com/paper/1602.07868/) (2016)
- [Large-Margin Softmax Loss for Convolutional Neural Networks](https://mlsanity.com/paper/1612.02295/) (2016)
- [Multimodal Compact Bilinear Pooling for Visual Question Answering and Visual Grounding](https://mlsanity.com/paper/1606.01847/) (2016)
- [Bilinear CNNs for Fine-grained Visual Recognition](https://mlsanity.com/paper/1504.07889/) (2015)
- [The MegaFace Benchmark: 1 Million Faces for Recognition at Scale](https://mlsanity.com/paper/1512.00596/) (2015)
- [Neural Photo Editing with Introspective Adversarial Networks](https://mlsanity.com/paper/1609.07093/) (2016)
- [Compact Bilinear Pooling](https://mlsanity.com/paper/1511.06062/) (2015)
- [Reducing Overfitting in Deep Networks by Decorrelating Representations](https://mlsanity.com/paper/1511.06068/) (2015)
- [Learning towards Minimum Hyperspherical Energy](https://mlsanity.com/paper/1805.09298/) (2018)
- [Deep Hyperspherical Learning](https://mlsanity.com/paper/1711.03189/) (2017)
- [Between-class Learning for Image Classification](https://mlsanity.com/paper/1711.10284/) (2017)
- [Dual Attention Matching Network for Context-Aware Feature Sequence based Person Re-Identification](https://mlsanity.com/paper/1803.09937/) (2018)
- [Decoupled Networks](https://mlsanity.com/paper/1804.08071/) (2018)
- [Neural Aggregation Network for Video Face Recognition](https://mlsanity.com/paper/1603.05474/) (2016)
- [Ring loss: Convex Feature Normalization for Face Recognition](https://mlsanity.com/paper/1803.00130/) (2018)
- [Quality Aware Network for Set to Set Recognition](https://mlsanity.com/paper/1704.03373/) (2017)
- [A Comprehensive Overview of Biometric Fusion](https://mlsanity.com/paper/1902.02919/) (2019)
- [Triplet Probabilistic Embedding for Face Verification and Clustering](https://mlsanity.com/paper/1604.05417/) (2016)
- [Template Adaptation for Face Verification and Identification](https://mlsanity.com/paper/1603.03958/) (2016)
- [Pairwise Relational Networks for Face Recognition](https://mlsanity.com/paper/1808.04976/) (2018)

## Built on by (most cited first, 2 of 2)

- [UFQA: Utility guided Fingerphoto Quality Assessment](https://mlsanity.com/paper/2407.11141/) (2024)
- [When Does Quality-Aware Multimodal Fusion Matter? A Leakage-Safe Diagnostic for Decision-Level Dependence](https://mlsanity.com/paper/2606.26473/) (2026)

## Abstract (arXiv)

We present a quality-aware multimodal recognition framework that combines representations from multiple biometric traits with varying quality and number of samples to achieve increased recognition accuracy by extracting complimentary identification information based on the quality of the samples. We develop a quality-aware framework for fusing representations of input modalities by weighting their importance using quality scores estimated in a weakly-supervised fashion. This framework utilizes two fusion blocks, each represented by a set of quality-aware and aggregation networks. In addition to architecture modifications, we propose two task-specific loss functions: multimodal separability loss and multimodal compactness loss. The first loss assures that the representations of modalities for a class have comparable magnitudes to provide a better quality estimation, while the multimodal representations of different classes are distributed to achieve maximum discrimination in the embedding space. The second loss, which is considered to regularize the network weights, improves the generalization performance by regularizing the framework. We evaluate the performance by considering three multimodal datasets consisting of face, iris, and fingerprint modalities. The efficacy of the framework is demonstrated through comparison with the state-of-the-art algorithms. In particular, our framework outperforms the rank- and score-level fusion of modalities of BIOMDATA by more than 30% for true acceptance rate at false acceptance rate of 10-4.