# Sphynx: ReLU-Efficient Network Design for Private Inference

arXiv: https://arxiv.org/abs/2106.11755 · HTML: https://mlsanity.com/paper/2106.11755/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 39 references that are in the map.
- First posted on arXiv 17 Jun 2021.

## Builds on (most cited first, 23 of 23)

- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [DARTS: Differentiable Architecture Search](https://mlsanity.com/paper/1806.09055/) (2018)
- [Neural Architecture Search with Reinforcement Learning](https://mlsanity.com/paper/1611.01578/) (2016)
- [Learning Transferable Architectures for Scalable Image Recognition](https://mlsanity.com/paper/1707.07012/) (2017)
- [Regularized Evolution for Image Classifier Architecture Search](https://mlsanity.com/paper/1802.01548/) (2018)
- [Efficient Neural Architecture Search via Parameter Sharing](https://mlsanity.com/paper/1802.03268/) (2018)
- [ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware](https://mlsanity.com/paper/1812.00332/) (2018)
- [Do Deep Nets Really Need to be Deep?](https://mlsanity.com/paper/1312.6184/) (2013)
- [BOHB: Robust and Efficient Hyperparameter Optimization at Scale](https://mlsanity.com/paper/1807.01774/) (2018)
- [SNAS: Stochastic Neural Architecture Search](https://mlsanity.com/paper/1812.09926/) (2018)
- [PC-DARTS: Partial Channel Connections for Memory-Efficient Architecture Search](https://mlsanity.com/paper/1907.05737/) (2019)
- [Searching for A Robust Neural Architecture in Four GPU Hours](https://mlsanity.com/paper/1910.04465/) (2019)
- [Random Search and Reproducibility for Neural Architecture Search](https://mlsanity.com/paper/1902.07638/) (2019)
- [Gazelle: A Low Latency Framework for Secure Neural Network Inference](https://mlsanity.com/paper/1801.05507/) (2018)
- [Hierarchical Representations for Efficient Architecture Search](https://mlsanity.com/paper/1711.00436/) (2017)
- [NAS evaluation is frustratingly hard](https://mlsanity.com/paper/1912.12522/) (2019)
- [XONN: XNOR-based Oblivious Deep Neural Network Inference](https://mlsanity.com/paper/1902.07342/) (2019)
- [DeepReDuce: ReLU Reduction for Fast Private Inference](https://mlsanity.com/paper/2103.01396/) (2021)
- [DeepSecure: Scalable Provably-Secure Deep Learning](https://mlsanity.com/paper/1705.08963/) (2017)
- [Geometry-Aware Gradient Algorithms for Neural Architecture Search](https://mlsanity.com/paper/2004.07802/) (2020)
- [CryptoNAS: Private Inference on a ReLU Budget](https://mlsanity.com/paper/2006.08733/) (2020)
- [One-Shot Neural Architecture Search via Compressive Sensing](https://mlsanity.com/paper/1906.02869/) (2019)

## Built on by (most cited first, 15 of 15)

- [Selective Network Linearization for Efficient Private Inference](https://mlsanity.com/paper/2202.02340/) (2022)
- [AutoReP: Automatic ReLU Replacement for Fast Private Network Inference](https://mlsanity.com/paper/2308.10134/) (2023)
- [MPCViT: Searching for Accurate and Efficient MPC-Friendly Vision Transformer with Heterogeneous Attention](https://mlsanity.com/paper/2211.13955/) (2022)
- [AESPA: Accuracy Preserving Low-degree Polynomial Activation for Fast Private Inference](https://mlsanity.com/paper/2201.06699/) (2022)
- [MPC-Minimized Secure LLM Inference](https://mlsanity.com/paper/2408.03561/) (2024)
- [Making Models Shallow Again: Jointly Learning to Reduce Non-Linearity and Depth for Latency-Efficient Private Inference](https://mlsanity.com/paper/2304.13274/) (2023)
- [PriViT: Vision Transformers for Fast Private Inference](https://mlsanity.com/paper/2310.04604/) (2023)
- [Tabula: Efficiently Computing Nonlinear Activation Functions for Secure Neural Network Inference](https://mlsanity.com/paper/2203.02833/) (2022)
- [Approximating ReLU on a Reduced Ring for Efficient MPC-based Private Inference](https://mlsanity.com/paper/2309.04875/) (2023)
- [HEQuant: Marrying Homomorphic Encryption and Quantization for Communication-Efficient Private Inference](https://mlsanity.com/paper/2401.15970/) (2024)
- [TruncFormer: Private LLM Inference Using Only Truncations](https://mlsanity.com/paper/2412.01042/) (2024)
- [Towards Fast and Scalable Private Inference](https://mlsanity.com/paper/2307.04077/) (2023)
- [CrypTorch: PyTorch-based Auto-tuning Compiler for Machine Learning with Multi-party Computation](https://mlsanity.com/paper/2511.19711/) (2025)
- [xMLP: Revolutionizing Private Inference with Exclusive Square Activation](https://mlsanity.com/paper/2403.08024/) (2024)
- [DeepShare: Sharing ReLU Across Channels and Layers for Efficient Private Inference](https://mlsanity.com/paper/2512.17398/) (2025)

## Abstract (arXiv)

The emergence of deep learning has been accompanied by privacy concerns surrounding users' data and service providers' models. We focus on private inference (PI), where the goal is to perform inference on a user's data sample using a service provider's model. Existing PI methods for deep networks enable cryptographically secure inference with little drop in functionality; however, they incur severe latency costs, primarily caused by non-linear network operations (such as ReLUs). This paper presents Sphynx, a ReLU-efficient network design method based on micro-search strategies for convolutional cell design. Sphynx achieves Pareto dominance over all existing private inference methods on CIFAR-100. We also design large-scale networks that support cryptographically private inference on Tiny-ImageNet and ImageNet.