# Privacy-Preserving Image Classification Using Vision Transformer

arXiv: https://arxiv.org/abs/2205.12041 · HTML: https://mlsanity.com/paper/2205.12041/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 32 references that are in the map.
- First posted on arXiv 24 May 2022.

## Builds on (most cited first, 14 of 14)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Identity Mappings in Deep Residual Networks](https://mlsanity.com/paper/1603.05027/) (2016)
- [ShakeDrop Regularization for Deep Residual Learning](https://mlsanity.com/paper/1802.02375/) (2018)
- [Encryption-then-Compression Systems using Grayscale-based Image Encryption for JPEG Images](https://mlsanity.com/paper/1811.00236/) (2018)
- [Learnable Image Encryption](https://mlsanity.com/paper/1804.00490/) (2018)
- [An Overview of Compressible and Learnable Image Transformation with Secret Key and Its Applications](https://mlsanity.com/paper/2201.11006/) (2022)
- [Block-wise Scrambled Image Recognition Using Adaptation Network](https://mlsanity.com/paper/2001.07761/) (2020)
- [A GAN-Based Image Transformation Scheme for Privacy-Preserving Deep Neural Networks](https://mlsanity.com/paper/2006.01342/) (2020)
- [Privacy-Preserving Image Classification Using Isotropic Network](https://mlsanity.com/paper/2204.07707/) (2022)
- [Attacks on Image Encryption Schemes for Privacy-Preserving Deep Neural Networks](https://mlsanity.com/paper/2004.13263/) (2020)
- [Image Transformation Network for Privacy-Preserving Deep Neural Networks and Its Security Evaluation](https://mlsanity.com/paper/2008.03143/) (2020)

## Built on by (most cited first, 9 of 9)

- [Image and Model Transformation with Secret Key for Vision Transformer](https://mlsanity.com/paper/2207.05366/) (2022)
- [Generative Model-Based Attack on Learnable Image Encryption for Privacy-Preserving Deep Learning](https://mlsanity.com/paper/2303.05036/) (2023)
- [Efficient Fine-Tuning with Domain Adaptation for Privacy-Preserving Vision Transformer](https://mlsanity.com/paper/2401.05126/) (2024)
- [A Random Ensemble of Encrypted Vision Transformers for Adversarially Robust Defense](https://mlsanity.com/paper/2402.07183/) (2024)
- [Color-NeuraCrypt: Privacy-Preserving Color-Image Classification Using Extended Random Neural Networks](https://mlsanity.com/paper/2301.04875/) (2023)
- [Preserving Data Privacy for ML-driven Applications in Open Radio Access Networks](https://mlsanity.com/paper/2402.09710/) (2024)
- [Domain Adaptation for Efficiently Fine-tuning Vision Transformer with Encrypted Images](https://mlsanity.com/paper/2309.02556/) (2023)
- [Privacy-Preserving Deep Learning Using Deformable Operators for Secure Task Learning](https://mlsanity.com/paper/2404.05828/) (2024)
- [A Privacy Preserving Method with a Random Orthogonal Matrix for ConvMixer Models](https://mlsanity.com/paper/2301.03843/) (2023)

## Abstract (arXiv)

In this paper, we propose a privacy-preserving image classification method that is based on the combined use of encrypted images and the vision transformer (ViT). The proposed method allows us not only to apply images without visual information to ViT models for both training and testing but to also maintain a high classification accuracy. ViT utilizes patch embedding and position embedding for image patches, so this architecture is shown to reduce the influence of block-wise image transformation. In an experiment, the proposed method for privacy-preserving image classification is demonstrated to outperform state-of-the-art methods in terms of classification accuracy and robustness against various attacks.