# AutoReP: Automatic ReLU Replacement for Fast Private Network Inference

arXiv: https://arxiv.org/abs/2308.10134 · HTML: https://mlsanity.com/paper/2308.10134/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Builds on 51 of its 90 references that are in the map.
- First posted on arXiv 20 Aug 2023.

## Builds on (most cited first, 30 of 51)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [Wide Residual Networks](https://mlsanity.com/paper/1605.07146/) (2016)
- [Pruning Filters for Efficient ConvNets](https://mlsanity.com/paper/1608.08710/) (2016)
- [SNIP: Single-shot Network Pruning based on Connection Sensitivity](https://mlsanity.com/paper/1810.02340/) (2018)
- [A Downsampled Variant of ImageNet as an Alternative to the CIFAR datasets](https://mlsanity.com/paper/1707.08819/) (2017)
- [Dynamic Network Surgery for Efficient DNNs](https://mlsanity.com/paper/1608.04493/) (2016)
- [Understanding Straight-Through Estimator in Training Activation Quantized Neural Nets](https://mlsanity.com/paper/1903.05662/) (2019)
- [Gazelle: A Low Latency Framework for Secure Neural Network Inference](https://mlsanity.com/paper/1801.05507/) (2018)
- [CrypTen: Secure Multi-Party Computation Meets Machine Learning](https://mlsanity.com/paper/2109.00984/) (2021)
- [Brain Network Transformer](https://mlsanity.com/paper/2210.06681/) (2022)
- [What Makes Convolutional Models Great on Long Sequence Modeling?](https://mlsanity.com/paper/2210.09298/) (2022)
- [Training Sparse Neural Networks](https://mlsanity.com/paper/1611.06694/) (2016)
- [FBNETGEN: Task-aware GNN-based fMRI Analysis via Functional Brain Network Generation](https://mlsanity.com/paper/2205.12465/) (2022)
- [Accelerating Dataset Distillation via Model Augmentation](https://mlsanity.com/paper/2212.06152/) (2022)
- [Low Latency Privacy Preserving Inference](https://mlsanity.com/paper/1812.10659/) (2018)
- [XONN: XNOR-based Oblivious Deep Neural Network Inference](https://mlsanity.com/paper/1902.07342/) (2019)
- [DeepReDuce: ReLU Reduction for Fast Private Inference](https://mlsanity.com/paper/2103.01396/) (2021)
- [CryptoNAS: Private Inference on a ReLU Budget](https://mlsanity.com/paper/2006.08733/) (2020)
- [Federated Contrastive Learning for Volumetric Medical Image Segmentation](https://mlsanity.com/paper/2204.10983/) (2022)
- [Against Membership Inference Attack: Pruning is All You Need](https://mlsanity.com/paper/2008.13578/) (2020)
- [Selective Network Linearization for Efficient Private Inference](https://mlsanity.com/paper/2202.02340/) (2022)
- [Accelerating Framework of Transformer by Hardware Design and Model Compression Co-Optimization](https://mlsanity.com/paper/2110.10030/) (2021)
- [Fast OSCAR and OWL Regression via Safe Screening Rules](https://mlsanity.com/paper/2006.16433/) (2020)
- [Enabling On-Device CNN Training by Self-Supervised Instance Filtering and Error Map Pruning](https://mlsanity.com/paper/2007.03213/) (2020)
- [Training Binary Neural Networks using the Bayesian Learning Rule](https://mlsanity.com/paper/2002.10778/) (2020)
- [A Length Adaptive Algorithm-Hardware Co-design of Transformer on FPGA Through Sparse Attention and Dynamic Pipelining](https://mlsanity.com/paper/2208.03646/) (2022)
- [Enabling On-Device Self-Supervised Contrastive Learning With Selective Data Contrast](https://mlsanity.com/paper/2106.03796/) (2021)
- [Zero-Shot Scene Graph Relation Prediction through Commonsense Knowledge Integration](https://mlsanity.com/paper/2107.05080/) (2021)
- [Sparse Progressive Distillation: Resolving Overfitting under Pretrain-and-Finetune Paradigm](https://mlsanity.com/paper/2110.08190/) (2021)

## Built on by (most cited first, 29 of 29)

- [Learning from Teaching Regularization: Generalizable Correlations Should be Easy to Imitate](https://mlsanity.com/paper/2402.02769/) (2024)
- [Time Series Modeling for Heart Rate Prediction: From ARIMA to Transformers](https://mlsanity.com/paper/2406.12199/) (2024)
- [Regional Style and Color Transfer](https://mlsanity.com/paper/2404.13880/) (2024)
- [APEER: Automatic Prompt Engineering Enhances Large Language Model Reranking](https://mlsanity.com/paper/2406.14449/) (2024)
- [LinGCN: Structural Linearized Graph Convolutional Network for Homomorphically Encrypted Inference](https://mlsanity.com/paper/2309.14331/) (2023)
- [MaxK-GNN: Extremely Fast GPU Kernel Design for Accelerating Graph Neural Networks Training](https://mlsanity.com/paper/2312.08656/) (2023)
- [A Comparative Study on Enhancing Prediction in Social Network Advertisement through Data Augmentation](https://mlsanity.com/paper/2404.13812/) (2024)
- [Enhance Image-to-Image Generation with LLaVA-generated Prompts](https://mlsanity.com/paper/2406.01956/) (2024)
- [Cross-border Commodity Pricing Strategy Optimization via Mixed Neural Network for Time Series Analysis](https://mlsanity.com/paper/2408.12115/) (2024)
- [Deep Analysis of Time Series Data for Smart Grid Startup Strategies: A Transformer-LSTM-PSO Model Approach](https://mlsanity.com/paper/2408.12129/) (2024)
- [Advanced AI Framework for Enhanced Detection and Assessment of Abdominal Trauma: Integrating 3D Segmentation with 2D CNN and RNN Models](https://mlsanity.com/paper/2407.16165/) (2024)
- [Muffin: A Framework Toward Multi-Dimension AI Fairness by Uniting Off-the-Shelf Models](https://mlsanity.com/paper/2308.13730/) (2023)
- [DLoRA: Distributed Parameter-Efficient Fine-Tuning Solution for Large Language Model](https://mlsanity.com/paper/2404.05182/) (2024)
- [Recording Brain Activity While Listening to Music Using Wearable EEG Devices Combined with Bidirectional Long Short-Term Memory Networks](https://mlsanity.com/paper/2408.12124/) (2024)
- [Privacy-Preserving Diffusion Model Using Homomorphic Encryption](https://mlsanity.com/paper/2403.05794/) (2024)
- [PristiQ: A Co-Design Framework for Preserving Data Security of Quantum Learning in the Cloud](https://mlsanity.com/paper/2404.13475/) (2024)
- [Optimized Layerwise Approximation for Efficient Private Inference on Fully Homomorphic Encryption](https://mlsanity.com/paper/2310.10349/) (2023)
- [Intelligent logistics management robot path planning algorithm integrating transformer and GCN network](https://mlsanity.com/paper/2501.02749/) (2025)
- [Edge-InversionNet: Enabling Efficient Inference of InversionNet on Edge Devices](https://mlsanity.com/paper/2310.09667/) (2023)
- [Data-Algorithm-Architecture Co-Optimization for Fair Neural Networks on Skin Lesion Dataset](https://mlsanity.com/paper/2407.13896/) (2024)
- [Key Information Retrieval to Classify the Unstructured Data Content of Preferential Trade Agreements](https://mlsanity.com/paper/2401.12520/) (2024)
- [Neural Networks with (Low-Precision) Polynomial Approximations: New Insights and Techniques for Accuracy Improvement](https://mlsanity.com/paper/2402.11224/) (2024)
- [A Self-guided Multimodal Approach to Enhancing Graph Representation Learning for Alzheimer's Diseases](https://mlsanity.com/paper/2412.06212/) (2024)
- [Regularized PolyKervNets: Optimizing Expressiveness and Efficiency for Private Inference in Deep Neural Networks](https://mlsanity.com/paper/2312.15229/) (2023)
- [International Trade Flow Prediction with Bilateral Trade Provisions](https://mlsanity.com/paper/2407.13698/) (2024)
- [Risk Analysis in Customer Relationship Management via Quantile Region Convolutional Neural Network-Long Short-Term Memory and Cross-Attention Mechanism](https://mlsanity.com/paper/2408.12113/) (2024)
- [Coordinate Descent for Network Linearization](https://mlsanity.com/paper/2511.11781/) (2025)
- [DeepShare: Sharing ReLU Across Channels and Layers for Efficient Private Inference](https://mlsanity.com/paper/2512.17398/) (2025)
- [UFO: Unlocking Ultra-Efficient Quantized Private Inference with Protocol and Algorithm Co-Optimization](https://mlsanity.com/paper/2602.18758/) (2026)

## Abstract (arXiv)

The growth of the Machine-Learning-As-A-Service (MLaaS) market has highlighted clients' data privacy and security issues. Private inference (PI) techniques using cryptographic primitives offer a solution but often have high computation and communication costs, particularly with non-linear operators like ReLU. Many attempts to reduce ReLU operations exist, but they may need heuristic threshold selection or cause substantial accuracy loss. This work introduces AutoReP, a gradient-based approach to lessen non-linear operators and alleviate these issues. It automates the selection of ReLU and polynomial functions to speed up PI applications and introduces distribution-aware polynomial approximation (DaPa) to maintain model expressivity while accurately approximating ReLUs. Our experimental results demonstrate significant accuracy improvements of 6.12% (94.31%, 12.9K ReLU budget, CIFAR-10), 8.39% (74.92%, 12.9K ReLU budget, CIFAR-100), and 9.45% (63.69%, 55K ReLU budget, Tiny-ImageNet) over current state-of-the-art methods, e.g., SNL. Morever, AutoReP is applied to EfficientNet-B2 on ImageNet dataset, and achieved 75.55% accuracy with 176.1 times ReLU budget reduction.