# Privacy-Preserving Vision Transformer Using Images Encrypted with Restricted Random Permutation Matrices

arXiv: https://arxiv.org/abs/2408.08529 · HTML: https://mlsanity.com/paper/2408.08529/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 9 of its 12 references that are in the map.
- First posted on arXiv 16 Aug 2024.

## Builds on (most cited first, 9 of 9)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [An Overview of Compressible and Learnable Image Transformation with Secret Key and Its Applications](https://mlsanity.com/paper/2201.11006/) (2022)
- [Block-wise Scrambled Image Recognition Using Adaptation Network](https://mlsanity.com/paper/2001.07761/) (2020)
- [Privacy-Preserving Image Classification Using Isotropic Network](https://mlsanity.com/paper/2204.07707/) (2022)
- [Image and Model Transformation with Secret Key for Vision Transformer](https://mlsanity.com/paper/2207.05366/) (2022)
- [Multi-Task Distributed Learning using Vision Transformer with Random Patch Permutation](https://mlsanity.com/paper/2204.03500/) (2022)
- [Grayscale-based Block Scrambling Image Encryption for Social Networking Services](https://mlsanity.com/paper/1806.03787/) (2018)
- [Efficient Fine-Tuning with Domain Adaptation for Privacy-Preserving Vision Transformer](https://mlsanity.com/paper/2401.05126/) (2024)
- [A Random Ensemble of Encrypted Vision Transformers for Adversarially Robust Defense](https://mlsanity.com/paper/2402.07183/) (2024)

## Abstract (arXiv)

We propose a novel method for privacy-preserving fine-tuning vision transformers (ViTs) with encrypted images. Conventional methods using encrypted images degrade model performance compared with that of using plain images due to the influence of image encryption. In contrast, the proposed encryption method using restricted random permutation matrices can provide a higher performance than the conventional ones.