Approximating ReLU on a Reduced Ring for Efficient MPC-based Private Inference
Not on the map. Why?
Abstract (arXiv)
Secure multi-party computation (MPC) allows users to offload machine learning inference on untrusted servers without having to share their privacy-sensitive data. Despite their strong security properties, MPC-based private inference has not been widely adopted in the real world due to their high communication overhead. When evaluating ReLU layers, MPC protocols incur a significant amount of communication between the parties, making the end-to-end execution time multiple orders slower than its non-private counterpart. This paper presents HummingBird, an MPC framework that reduces the ReLU communication overhead significantly by using only a subset of the bits to evaluate ReLU on a smaller ring. Based on theoretical analyses, HummingBird identifies bits in the secret share that are not crucial for accuracy and excludes them during ReLU evaluation to reduce communication. With its efficient search engine, HummingBird discards 87--91% of the bits during ReLU and still maintains high accuracy. On a real MPC setup involving multiple servers, HummingBird achieves on average 2.03--2.67x end-to-end speedup without introducing any errors, and up to 8.64x average speedup when some amount of accuracy degradation can be tolerated, due to its up to 8.76x communication reduction.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 50 of its 117 references that are in the map. First posted on arXiv 9 Sep 2023.
Builds on
50 of 117 references.
References · 50
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
- 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,291
- 2018Split learning for health: Distributed deep learning without sharing raw patient datacited by 391
- 2020SplitFed: When Federated Learning Meets Split Learningcited by 308
- 2018Gazelle: A Low Latency Framework for Secure Neural Network Inferencecited by 193
- 2021CrypTen: Secure Multi-Party Computation Meets Machine Learningcited by 131
- 2022The Lazy Neuron Phenomenon: On Emergence of Activation Sparsity in Transformerscited by 121
- 2020CrypTFlow2: Practical 2-Party Secure Inferencecited by 82
- 2020FALCON: Honest-Majority Maliciously Secure Framework for Private Deep Learningcited by 74
- 2019Split Learning for collaborative deep learning in healthcarecited by 72
- 2018Chameleon: A Hybrid Secure Computation Framework for Machine Learning Applicationscited by 67
38 more
- 2022MPCFormer: fast, performant and private Transformer inference with MPCcited by 67
- 2020NoPeek: Information leakage reduction to share activations in distributed deep learningcited by 66
- 2019CrypTFlow: Secure TensorFlow Inferencecited by 61
- 2021Memory-efficient Transformers via Top-k Attentioncited by 59
- 2021DeepReDuce: ReLU Reduction for Fast Private Inferencecited by 50
- 2021CryptGPU: Fast Privacy-Preserving Machine Learning on the GPUcited by 46
- 2022ResSFL: A Resistance Transfer Framework for Defending Model Inversion Attack in Split Federated Learningcited by 36
- 2020BLAZE: Blazing Fast Privacy-Preserving Machine Learningcited by 32
- 2022Selective Network Linearization for Efficient Private Inferencecited by 31
- 2021Practical Defences Against Model Inversion Attacks for Split Neural Networkscited by 29
- 2019Trident: Efficient 4PC Framework for Privacy Preserving Machine Learningcited by 26
- 2021F1: A Fast and Programmable Accelerator for Fully Homomorphic Encryption (Extended Version)cited by 23
- 2020Is Private Learning Possible with Instance Encoding?cited by 22
- 2023Learning to Linearize Deep Neural Networks for Secure and Efficient Private Inferencecited by 20
- 2019ASTRA: High Throughput 3PC over Rings with Application to Secure Predictioncited by 18
- 2017Leaky Cauldron on the Dark Land: Understanding Memory Side-Channel Hazards in SGXcited by 15
- 2021Sphynx: ReLU-Efficient Network Design for Private Inferencecited by 15
- 2021Sisyphus: A Cautionary Tale of Using Low-Degree Polynomial Activations in Privacy-Preserving Deep Learningcited by 14
- 2022AESPA: Accuracy Preserving Low-degree Polynomial Activation for Fast Private Inferencecited by 14
- 2021BTS: An Accelerator for Bootstrappable Fully Homomorphic Encryptioncited by 12
- 2019HEAX: An Architecture for Computing on Encrypted Datacited by 11
- 2021NeuraCrypt: Hiding Private Health Data via Random Neural Networks for Public Trainingcited by 11
- 2022ARK: Fully Homomorphic Encryption Accelerator with Runtime Data Generation and Inter-Operation Key Reusecited by 11
- 2023Bounding the Invertibility of Privacy-preserving Instance Encoding using Fisher Informationcited by 11
- 2019Interpretable Complex-Valued Neural Networks for Privacy Protectioncited by 10
- 2021NeuraCrypt is not privatecited by 10
- 2023HyPHEN: A Hybrid Packing Method and Optimizations for Homomorphic Encryption-Based Neural Networkscited by 9
- 2022Characterizing and Optimizing End-to-End Systems for Private Inferencecited by 8
- 2023DeepReShape: Redesigning Neural Networks for Efficient Private Inferencecited by 8
- 2022FAB: An FPGA-based Accelerator for Bootstrappable Fully Homomorphic Encryptioncited by 5
- 2023GPU-based Private Information Retrieval for On-Device Machine Learning Inferencecited by 5
- 2021ShEF: Shielded Enclaves for Cloud FPGAscited by 4
- 2021IceClave: A Trusted Execution Environment for In-Storage Computingcited by 3
- 2022Measuring and Controlling Split Layer Privacy Leakage Using Fisher Informationcited by 3
- 2020SEALing Neural Network Models in Secure Deep Learning Acceleratorscited by 2
- 2021Understanding TEE Containers, Easy to Use? Hard to Trustcited by 2
- 2018MI6: Secure Enclaves in a Speculative Out-of-Order Processorcited by 1
- 2021CryptoNite: Revealing the Pitfalls of End-to-End Private Inference at Scalecited by 1
Built on by
Citing papers per year: 2025: 1; 2026: 1.
2 papers.