# HEFT: Homomorphically Encrypted Fusion of Biometric Templates

arXiv: https://arxiv.org/abs/2208.07241 · HTML: https://mlsanity.com/paper/2208.07241/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 54 references that are in the map.
- First posted on arXiv 15 Aug 2022.

## Builds on (most cited first, 13 of 13)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [CosFace: Large Margin Cosine Loss for Deep Face Recognition](https://mlsanity.com/paper/1801.09414/) (2018)
- [Gazelle: A Low Latency Framework for Secure Neural Network Inference](https://mlsanity.com/paper/1801.05507/) (2018)
- [On the Reconstruction of Face Images from Deep Face Templates](https://mlsanity.com/paper/1703.00832/) (2017)
- [Deep Speaker: an End-to-End Neural Speaker Embedding System](https://mlsanity.com/paper/1705.02304/) (2017)
- [Learning a Fixed-Length Fingerprint Representation](https://mlsanity.com/paper/1909.09901/) (2019)
- [Secure Face Matching Using Fully Homomorphic Encryption](https://mlsanity.com/paper/1805.00577/) (2018)
- [HERS: Homomorphically Encrypted Representation Search](https://mlsanity.com/paper/2003.12197/) (2020)
- [Multibiometric Secure System Based on Deep Learning](https://mlsanity.com/paper/1708.02314/) (2017)
- [Feature Fusion Methods for Indexing and Retrieval of Biometric Data: Application to Face Recognition with Privacy Protection](https://mlsanity.com/paper/2107.12675/) (2021)
- [MobiBits: Multimodal Mobile Biometric Database](https://mlsanity.com/paper/1808.10710/) (2018)

## Built on by (most cited first, 4 of 4)

- [Privacy-preserving Multi-biometric Indexing based on Frequent Binary Patterns](https://mlsanity.com/paper/2310.03091/) (2023)
- [Multi-Biometric Fuzzy Vault based on Face and Fingerprints](https://mlsanity.com/paper/2301.06882/) (2023)
- [AMB-FHE: Adaptive Multi-biometric Fusion with Fully Homomorphic Encryption](https://mlsanity.com/paper/2503.23949/) (2025)
- [Multimodal Privacy-Preserving Entity Resolution with Fully Homomorphic Encryption](https://mlsanity.com/paper/2601.18612/) (2026)

## Abstract (arXiv)

This paper proposes a non-interactive end-to-end solution for secure fusion and matching of biometric templates using fully homomorphic encryption (FHE). Given a pair of encrypted feature vectors, we perform the following ciphertext operations, i) feature concatenation, ii) fusion and dimensionality reduction through a learned linear projection, iii) scale normalization to unit ℓ2-norm, and iv) match score computation. Our method, dubbed HEFT (Homomorphically Encrypted Fusion of biometric Templates), is custom-designed to overcome the unique constraint imposed by FHE, namely the lack of support for non-arithmetic operations. From an inference perspective, we systematically explore different data packing schemes for computationally efficient linear projection and introduce a polynomial approximation for scale normalization. From a training perspective, we introduce an FHE-aware algorithm for learning the linear projection matrix to mitigate errors induced by approximate normalization. Experimental evaluation for template fusion and matching of face and voice biometrics shows that HEFT (i) improves biometric verification performance by 11.07% and 9.58% AUROC compared to the respective unibiometric representations while compressing the feature vectors by a factor of 16 (512D to 32D), and (ii) fuses a pair of encrypted feature vectors and computes its match score against a gallery of size 1024 in 884 ms. Code and data are available at https://github.com/human-analysis/encrypted-biometric-fusion