# Efficient Fine-Tuning with Domain Adaptation for Privacy-Preserving Vision Transformer

arXiv: https://arxiv.org/abs/2401.05126 · HTML: https://mlsanity.com/paper/2401.05126/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 23 references that are in the map.
- First posted on arXiv 10 Jan 2024.

## Builds on (most cited first, 14 of 14)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Federated Learning: Strategies for Improving Communication Efficiency](https://mlsanity.com/paper/1610.05492/) (2016)
- [Encryption-then-Compression Systems using Grayscale-based Image Encryption for JPEG Images](https://mlsanity.com/paper/1811.00236/) (2018)
- [Learnable Image Encryption](https://mlsanity.com/paper/1804.00490/) (2018)
- [Block-wise Image Transformation with Secret Key for Adversarially Robust Defense](https://mlsanity.com/paper/2010.00801/) (2020)
- [An Overview of Compressible and Learnable Image Transformation with Secret Key and Its Applications](https://mlsanity.com/paper/2201.11006/) (2022)
- [Block-wise Scrambled Image Recognition Using Adaptation Network](https://mlsanity.com/paper/2001.07761/) (2020)
- [Encryption Inspired Adversarial Defense for Visual Classification](https://mlsanity.com/paper/2005.07998/) (2020)
- [Privacy-Preserving Image Classification Using Isotropic Network](https://mlsanity.com/paper/2204.07707/) (2022)
- [Attacks on Image Encryption Schemes for Privacy-Preserving Deep Neural Networks](https://mlsanity.com/paper/2004.13263/) (2020)
- [Image and Model Transformation with Secret Key for Vision Transformer](https://mlsanity.com/paper/2207.05366/) (2022)
- [NeuraCrypt: Hiding Private Health Data via Random Neural Networks for Public Training](https://mlsanity.com/paper/2106.02484/) (2021)
- [Privacy-Preserving Image Classification Using Vision Transformer](https://mlsanity.com/paper/2205.12041/) (2022)
- [Color-NeuraCrypt: Privacy-Preserving Color-Image Classification Using Extended Random Neural Networks](https://mlsanity.com/paper/2301.04875/) (2023)

## Built on by (most cited first, 3 of 3)

- [A Privacy-Preserving Semantic-Segmentation Method Using Domain-Adaptation Technique](https://mlsanity.com/paper/2507.12730/) (2025)
- [Privacy-Preserving Vision Transformer Using Images Encrypted with Restricted Random Permutation Matrices](https://mlsanity.com/paper/2408.08529/) (2024)
- [CFE-PPAR: Compression-friendly encryption for privacy-preserving action recognition leveraging video transformers](https://mlsanity.com/paper/2605.05692/) (2026)

## Abstract (arXiv)

We propose a novel method for privacy-preserving deep neural networks (DNNs) with the Vision Transformer (ViT). The method allows us not only to train models and test with visually protected images but to also avoid the performance degradation caused from the use of encrypted images, whereas conventional methods cannot avoid the influence of image encryption. A domain adaptation method is used to efficiently fine-tune ViT with encrypted images. In experiments, the method is demonstrated to outperform conventional methods in an image classification task on the CIFAR-10 and ImageNet datasets in terms of classification accuracy.