# Privacy-Preserving CNN Training with Transfer Learning: Multiclass Logistic Regression

arXiv: https://arxiv.org/abs/2304.03807 · HTML: https://mlsanity.com/paper/2304.03807/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 21 references that are in the map.
- First posted on arXiv 7 Apr 2023.

## Builds on (most cited first, 4 of 4)

- [Privacy-Preserving Logistic Regression Training with A Faster Gradient Variant](https://mlsanity.com/paper/2201.10838/) (2022)
- [Volley Revolver: A Novel Matrix-Encoding Method for Privacy-Preserving Neural Networks (Inference)](https://mlsanity.com/paper/2201.12577/) (2022)
- [Multinomial Logistic Regression Algorithms via Quadratic Gradient](https://mlsanity.com/paper/2208.06828/) (2022)
- [On Polynomial Approximation of Activation Function](https://mlsanity.com/paper/2202.00004/) (2022)

## Built on by (most cited first, 2 of 2)

- [Privacy-Preserving 3-Layer Neural Network Training](https://mlsanity.com/paper/2308.09531/) (2023)
- [LFFR: Logistic Function For (single-output) Regression](https://mlsanity.com/paper/2407.09955/) (2024)

## Abstract (arXiv)

In this paper, we present a practical solution to implement privacy-preserving CNN training based on mere Homomorphic Encryption (HE) technique. To our best knowledge, this is the first attempt successfully to crack this nut and no work ever before has achieved this goal. Several techniques combine to accomplish the task:: (1) with transfer learning, privacy-preserving CNN training can be reduced to homomorphic neural network training, or even multiclass logistic regression (MLR) training; (2) via a faster gradient variant called textttQuadratic Gradient, an enhanced gradient method for MLR with a state-of-the-art performance in convergence speed is applied in this work to achieve high performance; (3) we employ the thought of transformation in mathematics to transform approximating Softmax function in the encryption domain to the approximation of the Sigmoid function. A new type of loss function termed textttSquared Likelihood Error has been developed alongside to align with this change.; and (4) we use a simple but flexible matrix-encoding method named textttVolley Revolver to manage the data flow in the ciphertexts, which is the key factor to complete the whole homomorphic CNN training. The complete, runnable C++ code to implement our work can be found at: hrefhttps://github.com/petitioner/HE.CNNtrainingtexttthttps://github.com/petitioner/HE.CNNtraining. We select textttREGNETX400MF as our pre-trained model for transfer learning. We use the first 128 MNIST training images as training data and the whole MNIST testing dataset as the testing data. The client only needs to upload 6 ciphertexts to the cloud and it takes ~ 21 mins to perform 2 iterations on a cloud with 64 vCPUs, resulting in a precision of 21.49%.