# HERS: Homomorphically Encrypted Representation Search

arXiv: https://arxiv.org/abs/2003.12197 · HTML: https://mlsanity.com/paper/2003.12197/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Mar 2020.

## Built on by (most cited first, 26 of 26)

- [Matryoshka Representation Learning](https://mlsanity.com/paper/2205.13147/) (2022)
- [Biometrics: Trust, but Verify](https://mlsanity.com/paper/2105.06625/) (2021)
- [PrintsGAN: Synthetic Fingerprint Generator](https://mlsanity.com/paper/2201.03674/) (2022)
- [SpoofGAN: Synthetic Fingerprint Spoof Images](https://mlsanity.com/paper/2204.06498/) (2022)
- [Deep Face Fuzzy Vault: Implementation and Performance](https://mlsanity.com/paper/2102.02458/) (2021)
- [Biometric Template Protection for Neural-Network-based Face Recognition Systems: A Survey of Methods and Evaluation Techniques](https://mlsanity.com/paper/2110.05044/) (2021)
- [IronMask: Modular Architecture for Protecting Deep Face Template](https://mlsanity.com/paper/2104.02239/) (2021)
- [Minutiae-Guided Fingerprint Embeddings via Vision Transformers](https://mlsanity.com/paper/2210.13994/) (2022)
- [SlerpFace: Face Template Protection via Spherical Linear Interpolation](https://mlsanity.com/paper/2407.03043/) (2024)
- [FPGAN-Control: A Controllable Fingerprint Generator for Training with Synthetic Data](https://mlsanity.com/paper/2310.19024/) (2023)
- [Feature Fusion Methods for Indexing and Retrieval of Biometric Data: Application to Face Recognition with Privacy Protection](https://mlsanity.com/paper/2107.12675/) (2021)
- [Mobile Contactless Palmprint Recognition: Use of Multiscale, Multimodel Embeddings](https://mlsanity.com/paper/2401.08111/) (2024)
- [HEFT: Homomorphically Encrypted Fusion of Biometric Templates](https://mlsanity.com/paper/2208.07241/) (2022)
- [Enhancing Privacy in Face Analytics Using Fully Homomorphic Encryption](https://mlsanity.com/paper/2404.16255/) (2024)
- [IDFace: Face Template Protection for Efficient and Secure Identification](https://mlsanity.com/paper/2507.12050/) (2025)
- [Towards Privacy-Preserving, Real-Time and Lossless Feature Matching](https://mlsanity.com/paper/2208.00214/) (2022)
- [Privacy-preserving Multi-biometric Indexing based on Frequent Binary Patterns](https://mlsanity.com/paper/2310.03091/) (2023)
- [Non-Adaptive Adversarial Face Generation](https://mlsanity.com/paper/2507.12107/) (2025)
- [Securing Face and Fingerprint Templates in Humanitarian Biometric Systems](https://mlsanity.com/paper/2508.18415/) (2025)
- [CryptoFace: End-to-End Encrypted Face Recognition](https://mlsanity.com/paper/2509.00332/) (2025)
- [A Deeper Dive into the Irreversibility of PolyProtect: Making Protected Face Templates Harder to Invert](https://mlsanity.com/paper/2605.03857/) (2026)
- [Improving the Behaviour of Vision Transformers with Token-consistent Stochastic Layers](https://mlsanity.com/paper/2112.15111/) (2021)
- [FaceCloak: Learning to Protect Face Templates](https://mlsanity.com/paper/2504.06131/) (2025)
- [Secure and Scalable Face Retrieval via Cancelable Product Quantization](https://mlsanity.com/paper/2509.00781/) (2025)
- [Atlas: Efficient Verifiable Semantic Search](https://mlsanity.com/paper/2609.11841/) (2026)
- [Is it Possible to Generate Irreversible PolyProtected Templates from Face Embeddings using System-Specific Keys?](https://mlsanity.com/paper/2610.01385/) (2026)

## Abstract (arXiv)

We present a method to search for a probe (or query) image representation against a large gallery in the encrypted domain. We require that the probe and gallery images be represented in terms of a fixed-length representation, which is typical for representations obtained from learned networks. Our encryption scheme is agnostic to how the fixed-length representation is obtained and can therefore be applied to any fixed-length representation in any application domain. Our method, dubbed HERS (Homomorphically Encrypted Representation Search), operates by (i) compressing the representation towards its estimated intrinsic dimensionality with minimal loss of accuracy (ii) encrypting the compressed representation using the proposed fully homomorphic encryption scheme, and (iii) efficiently searching against a gallery of encrypted representations directly in the encrypted domain, without decrypting them. Numerical results on large galleries of face, fingerprint, and object datasets such as ImageNet show that, for the first time, accurate and fast image search within the encrypted domain is feasible at scale (500 seconds; 275× speed up over state-of-the-art for encrypted search against a gallery of 100 million). Code is available at https://github.com/human-analysis/hers-encrypted-image-search