# Domain Adaptation for Efficiently Fine-tuning Vision Transformer with Encrypted Images

arXiv: https://arxiv.org/abs/2309.02556 · HTML: https://mlsanity.com/paper/2309.02556/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 18 references that are in the map.
- First posted on arXiv 5 Sep 2023.

## Builds on (most cited first, 12 of 12)

- [Federated Learning: Strategies for Improving Communication Efficiency](https://mlsanity.com/paper/1610.05492/) (2016)
- [Encryption-then-Compression Systems using Grayscale-based Image Encryption for JPEG Images](https://mlsanity.com/paper/1811.00236/) (2018)
- [Learnable Image Encryption](https://mlsanity.com/paper/1804.00490/) (2018)
- [Block-wise Image Transformation with Secret Key for Adversarially Robust Defense](https://mlsanity.com/paper/2010.00801/) (2020)
- [An Overview of Compressible and Learnable Image Transformation with Secret Key and Its Applications](https://mlsanity.com/paper/2201.11006/) (2022)
- [Block-wise Scrambled Image Recognition Using Adaptation Network](https://mlsanity.com/paper/2001.07761/) (2020)
- [Privacy-Preserving Image Classification Using Isotropic Network](https://mlsanity.com/paper/2204.07707/) (2022)
- [Attacks on Image Encryption Schemes for Privacy-Preserving Deep Neural Networks](https://mlsanity.com/paper/2004.13263/) (2020)
- [Image and Model Transformation with Secret Key for Vision Transformer](https://mlsanity.com/paper/2207.05366/) (2022)
- [NeuraCrypt: Hiding Private Health Data via Random Neural Networks for Public Training](https://mlsanity.com/paper/2106.02484/) (2021)
- [Privacy-Preserving Image Classification Using Vision Transformer](https://mlsanity.com/paper/2205.12041/) (2022)
- [Color-NeuraCrypt: Privacy-Preserving Color-Image Classification Using Extended Random Neural Networks](https://mlsanity.com/paper/2301.04875/) (2023)

## Built on by (most cited first, 1 of 1)

- [A Random Ensemble of Encrypted Vision Transformers for Adversarially Robust Defense](https://mlsanity.com/paper/2402.07183/) (2024)

## Abstract (arXiv)

In recent years, deep neural networks (DNNs) trained with transformed data have been applied to various applications such as privacy-preserving learning, access control, and adversarial defenses. However, the use of transformed data decreases the performance of models. Accordingly, in this paper, we propose a novel method for fine-tuning models with transformed images under the use of the vision transformer (ViT). The proposed domain adaptation method does not cause the accuracy degradation of models, and it is carried out on the basis of the embedding structure of ViT. In experiments, we confirmed that the proposed method prevents accuracy degradation even when using encrypted images with the CIFAR-10 and CIFAR-100 datasets.