# Reducing ReLU Count for Privacy-Preserving CNN Speedup

arXiv: https://arxiv.org/abs/2101.11835 · HTML: https://mlsanity.com/paper/2101.11835/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 13 references that are in the map.
- First posted on arXiv 28 Jan 2021.

## Builds on (most cited first, 5 of 5)

- [Gazelle: A Low Latency Framework for Secure Neural Network Inference](https://mlsanity.com/paper/1801.05507/) (2018)
- [Chameleon: A Hybrid Secure Computation Framework for Machine Learning Applications](https://mlsanity.com/paper/1801.03239/) (2018)
- [DeepSecure: Scalable Provably-Secure Deep Learning](https://mlsanity.com/paper/1705.08963/) (2017)
- [FALCON: A Fourier Transform Based Approach for Fast and Secure Convolutional Neural Network Predictions](https://mlsanity.com/paper/1811.08257/) (2018)
- [Crypto-Oriented Neural Architecture Design](https://mlsanity.com/paper/1911.12322/) (2019)

## Built on by (most cited first, 3 of 3)

- [Verifiable and Provably Secure Machine Unlearning](https://mlsanity.com/paper/2210.09126/) (2022)
- [SoK: Privacy-preserving Deep Learning with Homomorphic Encryption](https://mlsanity.com/paper/2112.12855/) (2021)
- [Securing Neural Networks with Knapsack Optimization](https://mlsanity.com/paper/2304.10442/) (2023)

## Abstract (arXiv)

Privacy-Preserving Machine Learning algorithms must balance classification accuracy with data privacy. This can be done using a combination of cryptographic and machine learning tools such as Convolutional Neural Networks (CNN). CNNs typically consist of two types of operations: a convolutional or linear layer, followed by a non-linear function such as ReLU. Each of these types can be implemented efficiently using a different cryptographic tool. But these tools require different representations and switching between them is time-consuming and expensive. Recent research suggests that ReLU is responsible for most of the communication bandwidth. ReLU is usually applied at each pixel (or activation) location, which is quite expensive. We propose to share ReLU operations. Specifically, the ReLU decision of one activation can be used by others, and we explore different ways to group activations and different ways to determine the ReLU for such a group of activations. Experiments on several datasets reveal that we can cut the number of ReLU operations by up to three orders of magnitude and, as a result, cut the communication bandwidth by more than 50%.