# RRNet: Towards ReLU-Reduced Neural Network for Two-party Computation Based Private Inference

arXiv: https://arxiv.org/abs/2302.02292 · HTML: https://mlsanity.com/paper/2302.02292/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 43 references that are in the map.
- First posted on arXiv 5 Feb 2023.

## Builds on (most cited first, 25 of 25)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [DARTS: Differentiable Architecture Search](https://mlsanity.com/paper/1806.09055/) (2018)
- [CrypTen: Secure Multi-Party Computation Meets Machine Learning](https://mlsanity.com/paper/2109.00984/) (2021)
- [Brain Network Transformer](https://mlsanity.com/paper/2210.06681/) (2022)
- [FBNETGEN: Task-aware GNN-based fMRI Analysis via Functional Brain Network Generation](https://mlsanity.com/paper/2205.12465/) (2022)
- [CrypTFlow: Secure TensorFlow Inference](https://mlsanity.com/paper/1909.07814/) (2019)
- [DeepReDuce: ReLU Reduction for Fast Private Inference](https://mlsanity.com/paper/2103.01396/) (2021)
- [CryptGPU: Fast Privacy-Preserving Machine Learning on the GPU](https://mlsanity.com/paper/2104.10949/) (2021)
- [Co-Exploration of Neural Architectures and Heterogeneous ASIC Accelerator Designs Targeting Multiple Tasks](https://mlsanity.com/paper/2002.04116/) (2020)
- [CryptoNAS: Private Inference on a ReLU Budget](https://mlsanity.com/paper/2006.08733/) (2020)
- [Federated Contrastive Learning for Volumetric Medical Image Segmentation](https://mlsanity.com/paper/2204.10983/) (2022)
- [Selective Network Linearization for Efficient Private Inference](https://mlsanity.com/paper/2202.02340/) (2022)
- [Enabling On-Device CNN Training by Self-Supervised Instance Filtering and Error Map Pruning](https://mlsanity.com/paper/2007.03213/) (2020)
- [A Length Adaptive Algorithm-Hardware Co-design of Transformer on FPGA Through Sparse Attention and Dynamic Pipelining](https://mlsanity.com/paper/2208.03646/) (2022)
- [Zero-Shot Scene Graph Relation Prediction through Commonsense Knowledge Integration](https://mlsanity.com/paper/2107.05080/) (2021)
- [Federated Contrastive Learning for Dermatological Disease Diagnosis via On-device Learning](https://mlsanity.com/paper/2202.07470/) (2022)
- [Intermittent Inference with Nonuniformly Compressed Multi-Exit Neural Network for Energy Harvesting Powered Devices](https://mlsanity.com/paper/2004.11293/) (2020)
- [Towards Sparsification of Graph Neural Networks](https://mlsanity.com/paper/2209.04766/) (2022)
- [Dynamic Sparse Training via Balancing the Exploration-Exploitation Trade-off](https://mlsanity.com/paper/2211.16667/) (2022)
- [An Accelerated Doubly Stochastic Gradient Method with Faster Explicit Model Identification](https://mlsanity.com/paper/2208.06058/) (2022)
- [An Automatic and Efficient BERT Pruning for Edge AI Systems](https://mlsanity.com/paper/2206.10461/) (2022)
- [Improving DNN Fault Tolerance using Weight Pruning and Differential Crossbar Mapping for ReRAM-based Edge AI](https://mlsanity.com/paper/2106.09166/) (2021)
- [RADARS: Memory Efficient Reinforcement Learning Aided Differentiable Neural Architecture Search](https://mlsanity.com/paper/2109.05691/) (2021)

## Built on by (most cited first, 12 of 12)

- [Learning from Teaching Regularization: Generalizable Correlations Should be Easy to Imitate](https://mlsanity.com/paper/2402.02769/) (2024)
- [LinGCN: Structural Linearized Graph Convolutional Network for Homomorphically Encrypted Inference](https://mlsanity.com/paper/2309.14331/) (2023)
- [MaxK-GNN: Extremely Fast GPU Kernel Design for Accelerating Graph Neural Networks Training](https://mlsanity.com/paper/2312.08656/) (2023)
- [Predict Click-Through Rates with Deep Interest Network Model in E-commerce Advertising](https://mlsanity.com/paper/2406.10239/) (2024)
- [Optimizing Search Advertising Strategies: Integrating Reinforcement Learning with Generalized Second-Price Auctions for Enhanced Ad Ranking and Bidding](https://mlsanity.com/paper/2405.13381/) (2024)
- [DLoRA: Distributed Parameter-Efficient Fine-Tuning Solution for Large Language Model](https://mlsanity.com/paper/2404.05182/) (2024)
- [QuMoS: A Framework for Preserving Security of Quantum Machine Learning Model](https://mlsanity.com/paper/2304.11511/) (2023)
- [MPC-Minimized Secure LLM Inference](https://mlsanity.com/paper/2408.03561/) (2024)
- [PristiQ: A Co-Design Framework for Preserving Data Security of Quantum Learning in the Cloud](https://mlsanity.com/paper/2404.13475/) (2024)
- [PriViT: Vision Transformers for Fast Private Inference](https://mlsanity.com/paper/2310.04604/) (2023)
- [Key Information Retrieval to Classify the Unstructured Data Content of Preferential Trade Agreements](https://mlsanity.com/paper/2401.12520/) (2024)
- [International Trade Flow Prediction with Bilateral Trade Provisions](https://mlsanity.com/paper/2407.13698/) (2024)

## Abstract (arXiv)

The proliferation of deep learning (DL) has led to the emergence of privacy and security concerns. To address these issues, secure Two-party computation (2PC) has been proposed as a means of enabling privacy-preserving DL computation. However, in practice, 2PC methods often incur high computation and communication overhead, which can impede their use in large-scale systems. To address this challenge, we introduce RRNet, a systematic framework that aims to jointly reduce the overhead of MPC comparison protocols and accelerate computation through hardware acceleration. Our approach integrates the hardware latency of cryptographic building blocks into the DNN loss function, resulting in improved energy efficiency, accuracy, and security guarantees. Furthermore, we propose a cryptographic hardware scheduler and corresponding performance model for Field Programmable Gate Arrays (FPGAs) to further enhance the efficiency of our framework. Experiments show RRNet achieved a much higher ReLU reduction performance than all SOTA works on CIFAR-10 dataset.