# Image and Model Transformation with Secret Key for Vision Transformer

arXiv: https://arxiv.org/abs/2207.05366 · HTML: https://mlsanity.com/paper/2207.05366/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 61 references that are in the map.
- First posted on arXiv 12 Jul 2022.

## Builds on (most cited first, 30 of 30)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Membership Inference Attacks against Machine Learning Models](https://mlsanity.com/paper/1610.05820/) (2016)
- [MLP-Mixer: An all-MLP Architecture for Vision](https://mlsanity.com/paper/2105.01601/) (2021)
- [Pay Attention to MLPs](https://mlsanity.com/paper/2105.08050/) (2021)
- [Turning Your Weakness Into a Strength: Watermarking Deep Neural Networks by Backdooring](https://mlsanity.com/paper/1802.04633/) (2018)
- [ResMLP: Feedforward networks for image classification with data-efficient training](https://mlsanity.com/paper/2105.03404/) (2021)
- [Embedding Watermarks into Deep Neural Networks](https://mlsanity.com/paper/1701.04082/) (2017)
- [Patches Are All You Need?](https://mlsanity.com/paper/2201.09792/) (2022)
- [CycleMLP: A MLP-like Architecture for Dense Prediction](https://mlsanity.com/paper/2107.10224/) (2021)
- [Adversarial Machine Learning -- Industry Perspectives](https://mlsanity.com/paper/2002.05646/) (2020)
- [Adversarial Frontier Stitching for Remote Neural Network Watermarking](https://mlsanity.com/paper/1711.01894/) (2017)
- [Vision Permutator: A Permutable MLP-Like Architecture for Visual Recognition](https://mlsanity.com/paper/2106.12368/) (2021)
- [Recent progress in semantic image segmentation](https://mlsanity.com/paper/1809.10198/) (2018)
- [Encryption-then-Compression Systems using Grayscale-based Image Encryption for JPEG Images](https://mlsanity.com/paper/1811.00236/) (2018)
- [Learnable Image Encryption](https://mlsanity.com/paper/1804.00490/) (2018)
- [Block-wise Image Transformation with Secret Key for Adversarially Robust Defense](https://mlsanity.com/paper/2010.00801/) (2020)
- [An Overview of Compressible and Learnable Image Transformation with Secret Key and Its Applications](https://mlsanity.com/paper/2201.11006/) (2022)
- [A GAN-Based Image Transformation Scheme for Privacy-Preserving Deep Neural Networks](https://mlsanity.com/paper/2006.01342/) (2020)
- [Privacy-Preserving Deep Neural Networks with Pixel-based Image Encryption Considering Data Augmentation in the Encrypted Domain](https://mlsanity.com/paper/1905.01827/) (2019)
- [Encryption Inspired Adversarial Defense for Visual Classification](https://mlsanity.com/paper/2005.07998/) (2020)
- [Privacy-Preserving Image Classification Using Isotropic Network](https://mlsanity.com/paper/2204.07707/) (2022)
- [A Protection Method of Trained CNN Model with Secret Key from Unauthorized Access](https://mlsanity.com/paper/2105.14756/) (2021)
- [A Privacy-Preserving Machine Learning Scheme Using EtC Images](https://mlsanity.com/paper/2007.08775/) (2020)
- [Privacy-Preserving Image Classification Using Vision Transformer](https://mlsanity.com/paper/2205.12041/) (2022)
- [P3: Toward Privacy-Preserving Photo Sharing](https://mlsanity.com/paper/1302.5062/) (2013)
- [Privacy-Preserving Support Vector Machine Computing Using Random Unitary Transformation](https://mlsanity.com/paper/1908.07915/) (2019)
- [Block shuffling learning for Deepfake Detection](https://mlsanity.com/paper/2202.02819/) (2022)
- [A Reversible Data Hiding Method in Compressible Encrypted Images](https://mlsanity.com/paper/2011.08969/) (2020)

## Built on by (most cited first, 12 of 12)

- [Efficient Fine-Tuning with Domain Adaptation for Privacy-Preserving Vision Transformer](https://mlsanity.com/paper/2401.05126/) (2024)
- [Color-NeuraCrypt: Privacy-Preserving Color-Image Classification Using Extended Random Neural Networks](https://mlsanity.com/paper/2301.04875/) (2023)
- [Domain Adaptation for Efficiently Fine-tuning Vision Transformer with Encrypted Images](https://mlsanity.com/paper/2309.02556/) (2023)
- [A Privacy-Preserving Semantic-Segmentation Method Using Domain-Adaptation Technique](https://mlsanity.com/paper/2507.12730/) (2025)
- [Keys in the Weights: Transformer Authentication Using Model-Bound Latent Representations](https://mlsanity.com/paper/2511.00973/) (2025)
- [Privacy-Preserving Image Classification Using ConvMixer with Adaptive Permutation Matrix](https://mlsanity.com/paper/2208.02556/) (2022)
- [An Access Control Method with Secret Key for Semantic Segmentation Models](https://mlsanity.com/paper/2208.13135/) (2022)
- [A Privacy Preserving Method with a Random Orthogonal Matrix for ConvMixer Models](https://mlsanity.com/paper/2301.03843/) (2023)
- [Combined Use of Federated Learning and Image Encryption for Privacy-Preserving Image Classification with Vision Transformer](https://mlsanity.com/paper/2301.09255/) (2023)
- [XNN: Paradigm Shift in Mitigating Identity Leakage within Cloud-Enabled Deep Learning](https://mlsanity.com/paper/2408.04974/) (2024)
- [Privacy-Preserving Vision Transformer Using Images Encrypted with Restricted Random Permutation Matrices](https://mlsanity.com/paper/2408.08529/) (2024)
- [Privacy-Preserving Clothing Classification using Vision Transformer for Thermal Comfort Estimation](https://mlsanity.com/paper/2604.26184/) (2026)

## Abstract (arXiv)

In this paper, we propose a combined use of transformed images and vision transformer (ViT) models transformed with a secret key. We show for the first time that models trained with plain images can be directly transformed to models trained with encrypted images on the basis of the ViT architecture, and the performance of the transformed models is the same as models trained with plain images when using test images encrypted with the key. In addition, the proposed scheme does not require any specially prepared data for training models or network modification, so it also allows us to easily update the secret key. In an experiment, the effectiveness of the proposed scheme is evaluated in terms of performance degradation and model protection performance in an image classification task on the CIFAR-10 dataset.