# Privacy-Preserving Visual Learning Using Doubly Permuted Homomorphic Encryption

arXiv: https://arxiv.org/abs/1704.02203 · HTML: https://mlsanity.com/paper/1704.02203/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Apr 2017.

## Built on by (most cited first, 14 of 14)

- [A Study of Face Obfuscation in ImageNet](https://mlsanity.com/paper/2103.06191/) (2021)
- [Privacy-Preserving Portrait Matting](https://mlsanity.com/paper/2104.14222/) (2021)
- [Biometrics: Trust, but Verify](https://mlsanity.com/paper/2105.06625/) (2021)
- [Undistillable: Making A Nasty Teacher That CANNOT teach students](https://mlsanity.com/paper/2105.07381/) (2021)
- [Rethinking Portrait Matting with Privacy Preserving](https://mlsanity.com/paper/2203.16828/) (2022)
- [NinjaDesc: Content-Concealing Visual Descriptors via Adversarial Learning](https://mlsanity.com/paper/2112.12785/) (2021)
- [Analysis and Mitigations of Reverse Engineering Attacks on Local Feature Descriptors](https://mlsanity.com/paper/2105.03812/) (2021)
- [A Review on Visual Privacy Preservation Techniques for Active and Assisted Living](https://mlsanity.com/paper/2112.09422/) (2021)
- [Privacy-Preserving Diffusion Model Using Homomorphic Encryption](https://mlsanity.com/paper/2403.05794/) (2024)
- [Multitask Identity-Aware Image Steganography via Minimax Optimization](https://mlsanity.com/paper/2107.05819/) (2021)
- [IPProtect: protecting the intellectual property of visual datasets during data valuation](https://mlsanity.com/paper/2212.11468/) (2022)
- [Computation-Efficient and Recognition-Friendly 3D Point Cloud Privacy Protection](https://mlsanity.com/paper/2503.15818/) (2025)
- [Learning Private Representations through Entropy-based Adversarial Training](https://mlsanity.com/paper/2507.10194/) (2025)
- [Semantic Privacy Protection with Utility Preservation for 3D Point Clouds](https://mlsanity.com/paper/2609.13823/) (2026)

## Abstract (arXiv)

We propose a privacy-preserving framework for learning visual classifiers by leveraging distributed private image data. This framework is designed to aggregate multiple classifiers updated locally using private data and to ensure that no private information about the data is exposed during and after its learning procedure. We utilize a homomorphic cryptosystem that can aggregate the local classifiers while they are encrypted and thus kept secret. To overcome the high computational cost of homomorphic encryption of high-dimensional classifiers, we (1) impose sparsity constraints on local classifier updates and (2) propose a novel efficient encryption scheme named doubly-permuted homomorphic encryption (DPHE) which is tailored to sparse high-dimensional data. DPHE (i) decomposes sparse data into its constituent non-zero values and their corresponding support indices, (ii) applies homomorphic encryption only to the non-zero values, and (iii) employs double permutations on the support indices to make them secret. Our experimental evaluation on several public datasets shows that the proposed approach achieves comparable performance against state-of-the-art visual recognition methods while preserving privacy and significantly outperforms other privacy-preserving methods.