# Generative Model-Based Attack on Learnable Image Encryption for Privacy-Preserving Deep Learning

arXiv: https://arxiv.org/abs/2303.05036 · HTML: https://mlsanity.com/paper/2303.05036/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 57 references that are in the map.
- First posted on arXiv 9 Mar 2023.

## Builds on (most cited first, 30 of 34)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models](https://mlsanity.com/paper/2112.10741/) (2021)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [Deep Learning with Differential Privacy](https://mlsanity.com/paper/1607.00133/) (2016)
- [Image Super-Resolution via Iterative Refinement](https://mlsanity.com/paper/2104.07636/) (2021)
- [Federated Learning: Strategies for Improving Communication Efficiency](https://mlsanity.com/paper/1610.05492/) (2016)
- [Encoding in Style: a StyleGAN Encoder for Image-to-Image Translation](https://mlsanity.com/paper/2008.00951/) (2020)
- [Enabling Fast Differentially Private SGD via Just-in-Time Compilation and Vectorization](https://mlsanity.com/paper/2010.09063/) (2020)
- [Encryption-then-Compression Systems using Grayscale-based Image Encryption for JPEG Images](https://mlsanity.com/paper/1811.00236/) (2018)
- [Learnable Image Encryption](https://mlsanity.com/paper/1804.00490/) (2018)
- [Block-wise Image Transformation with Secret Key for Adversarially Robust Defense](https://mlsanity.com/paper/2010.00801/) (2020)
- [Fast and Memory Efficient Differentially Private-SGD via JL Projections](https://mlsanity.com/paper/2102.03013/) (2021)
- [An Overview of Compressible and Learnable Image Transformation with Secret Key and Its Applications](https://mlsanity.com/paper/2201.11006/) (2022)
- [Block-wise Scrambled Image Recognition Using Adaptation Network](https://mlsanity.com/paper/2001.07761/) (2020)
- [Privacy-Preserving Image Classification Using Isotropic Network](https://mlsanity.com/paper/2204.07707/) (2022)
- [A Protection Method of Trained CNN Model with Secret Key from Unauthorized Access](https://mlsanity.com/paper/2105.14756/) (2021)
- [Attacks on Image Encryption Schemes for Privacy-Preserving Deep Neural Networks](https://mlsanity.com/paper/2004.13263/) (2020)
- [Secure Collaborative Training and Inference for XGBoost](https://mlsanity.com/paper/2010.02524/) (2020)
- [Privacy-Preserving Image Classification Using Vision Transformer](https://mlsanity.com/paper/2205.12041/) (2022)

## Built on by (most cited first, 4 of 4)

- [Enhancing Deep Reinforcement Learning: A Tutorial on Generative Diffusion Models in Network Optimization](https://mlsanity.com/paper/2308.05384/) (2023)
- [Unveiling Hidden Visual Information: A Reconstruction Attack Against Adversarial Visual Information Hiding](https://mlsanity.com/paper/2408.04261/) (2024)
- [AI Security in the Foundation Model Era: A Comprehensive Survey from a Unified Perspective](https://mlsanity.com/paper/2603.24857/) (2026)
- [Attack GAN (AGAN ): A new Security Evaluation Tool for Perceptual Encryption](https://mlsanity.com/paper/2407.06570/) (2024)

## Abstract (arXiv)

In this paper, we propose a novel generative model-based attack on learnable image encryption methods proposed for privacy-preserving deep learning. Various learnable encryption methods have been studied to protect the sensitive visual information of plain images, and some of them have been investigated to be robust enough against all existing attacks. However, previous attacks on image encryption focus only on traditional cryptanalytic attacks or reverse translation models, so these attacks cannot recover any visual information if a block-scrambling encryption step, which effectively destroys global information, is applied. Accordingly, in this paper, generative models are explored to evaluate whether such models can restore sensitive visual information from encrypted images for the first time. We first point out that encrypted images have some similarity with plain images in the embedding space. By taking advantage of leaked information from encrypted images, we propose a guided generative model as an attack on learnable image encryption to recover personally identifiable visual information. We implement the proposed attack in two ways by utilizing two state-of-the-art generative models: a StyleGAN-based model and latent diffusion-based one. Experiments were carried out on the CelebA-HQ and ImageNet datasets. Results show that images reconstructed by the proposed method have perceptual similarities to plain images.