# A Protection Method of Trained CNN Model with Secret Key from Unauthorized Access

arXiv: https://arxiv.org/abs/2105.14756 · HTML: https://mlsanity.com/paper/2105.14756/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 39 references that are in the map.
- First posted on arXiv 31 May 2021.

## Builds on (most cited first, 23 of 23)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Mixed Precision Training](https://mlsanity.com/paper/1710.03740/) (2017)
- [Fast is better than free: Revisiting adversarial training](https://mlsanity.com/paper/2001.03994/) (2020)
- [Turning Your Weakness Into a Strength: Watermarking Deep Neural Networks by Backdooring](https://mlsanity.com/paper/1802.04633/) (2018)
- [Embedding Watermarks into Deep Neural Networks](https://mlsanity.com/paper/1701.04082/) (2017)
- [Super-Convergence: Very Fast Training of Neural Networks Using Large Learning Rates](https://mlsanity.com/paper/1708.07120/) (2017)
- [Adversarial Frontier Stitching for Remote Neural Network Watermarking](https://mlsanity.com/paper/1711.01894/) (2017)
- [[Extended version] Rethinking Deep Neural Network Ownership Verification: Embedding Passports to Defeat Ambiguity Attacks](https://mlsanity.com/paper/1909.07830/) (2019)
- [Encryption-then-Compression Systems using Grayscale-based Image Encryption for JPEG Images](https://mlsanity.com/paper/1811.00236/) (2018)
- [Learnable Image Encryption](https://mlsanity.com/paper/1804.00490/) (2018)
- [Block-wise Image Transformation with Secret Key for Adversarially Robust Defense](https://mlsanity.com/paper/2010.00801/) (2020)
- [Block-wise Scrambled Image Recognition Using Adaptation Network](https://mlsanity.com/paper/2001.07761/) (2020)
- [A GAN-Based Image Transformation Scheme for Privacy-Preserving Deep Neural Networks](https://mlsanity.com/paper/2006.01342/) (2020)
- [DeepMarks: A Digital Fingerprinting Framework for Deep Neural Networks](https://mlsanity.com/paper/1804.03648/) (2018)
- [Privacy-Preserving Deep Neural Networks with Pixel-based Image Encryption Considering Data Augmentation in the Encrypted Domain](https://mlsanity.com/paper/1905.01827/) (2019)
- [Encryption Inspired Adversarial Defense for Visual Classification](https://mlsanity.com/paper/2005.07998/) (2020)
- [Training DNN Model with Secret Key for Model Protection](https://mlsanity.com/paper/2008.02450/) (2020)
- [A Privacy-Preserving Machine Learning Scheme Using EtC Images](https://mlsanity.com/paper/2007.08775/) (2020)
- [Image Transformation Network for Privacy-Preserving Deep Neural Networks and Its Security Evaluation](https://mlsanity.com/paper/2008.03143/) (2020)

## Built on by (most cited first, 15 of 15)

- [An Overview of Compressible and Learnable Image Transformation with Secret Key and Its Applications](https://mlsanity.com/paper/2201.11006/) (2022)
- [Image and Model Transformation with Secret Key for Vision Transformer](https://mlsanity.com/paper/2207.05366/) (2022)
- [A Protection Method of Trained CNN Model Using Feature Maps Transformed With Secret Key From Unauthorized Access](https://mlsanity.com/paper/2109.00224/) (2021)
- [Access Control Using Spatially Invariant Permutation of Feature Maps for Semantic Segmentation Models](https://mlsanity.com/paper/2109.01332/) (2021)
- [Generative Model-Based Attack on Learnable Image Encryption for Privacy-Preserving Deep Learning](https://mlsanity.com/paper/2303.05036/) (2023)
- [Access Control of Semantic Segmentation Models Using Encrypted Feature Maps](https://mlsanity.com/paper/2206.05422/) (2022)
- [Hindering Adversarial Attacks with Implicit Neural Representations](https://mlsanity.com/paper/2210.13982/) (2022)
- [Access Control of Object Detection Models Using Encrypted Feature Maps](https://mlsanity.com/paper/2202.00265/) (2022)
- [Access Control with Encrypted Feature Maps for Object Detection Models](https://mlsanity.com/paper/2209.14831/) (2022)
- [A Privacy-Preserving Semantic-Segmentation Method Using Domain-Adaptation Technique](https://mlsanity.com/paper/2507.12730/) (2025)
- [Protecting Semantic Segmentation Models by Using Block-wise Image Encryption with Secret Key from Unauthorized Access](https://mlsanity.com/paper/2107.09362/) (2021)
- [Protection of SVM Model with Secret Key from Unauthorized Access](https://mlsanity.com/paper/2111.08927/) (2021)
- [An Encryption Method of ConvMixer Models without Performance Degradation](https://mlsanity.com/paper/2207.11939/) (2022)
- [An Access Control Method with Secret Key for Semantic Segmentation Models](https://mlsanity.com/paper/2208.13135/) (2022)
- [Privacy-Preserving Object Detection for Vision Transformer-Based Models](https://mlsanity.com/paper/2608.20712/) (2026)

## Abstract (arXiv)

In this paper, we propose a novel method for protecting convolutional neural network (CNN) models with a secret key set so that unauthorized users without the correct key set cannot access trained models. The method enables us to protect not only from copyright infringement but also the functionality of a model from unauthorized access without any noticeable overhead. We introduce three block-wise transformations with a secret key set to generate learnable transformed images: pixel shuffling, negative/positive transformation, and FFX encryption. Protected models are trained by using transformed images. The results of experiments with the CIFAR and ImageNet datasets show that the performance of a protected model was close to that of non-protected models when the key set was correct, while the accuracy severely dropped when an incorrect key set was given. The protected model was also demonstrated to be robust against various attacks. Compared with the state-of-the-art model protection with passports, the proposed method does not have any additional layers in the network, and therefore, there is no overhead during training and inference processes.