# Practical and Private Hybrid ML Inference with Fully Homomorphic Encryption

arXiv: https://arxiv.org/abs/2509.01253 · HTML: https://mlsanity.com/paper/2509.01253/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 47 references that are in the map.
- First posted on arXiv 1 Sep 2025.

## Builds on (most cited first, 16 of 16)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Gazelle: A Low Latency Framework for Secure Neural Network Inference](https://mlsanity.com/paper/1801.05507/) (2018)
- [Amplification by Shuffling: From Local to Central Differential Privacy via Anonymity](https://mlsanity.com/paper/1811.12469/) (2018)
- [Distributed Differential Privacy via Shuffling](https://mlsanity.com/paper/1808.01394/) (2018)
- [Hiding Among the Clones: A Simple and Nearly Optimal Analysis of Privacy Amplification by Shuffling](https://mlsanity.com/paper/2012.12803/) (2020)
- [The Privacy Blanket of the Shuffle Model](https://mlsanity.com/paper/1903.02837/) (2019)
- [Low Latency Privacy Preserving Inference](https://mlsanity.com/paper/1812.10659/) (2018)
- [Privacy-Preserving Machine Learning with Fully Homomorphic Encryption for Deep Neural Network](https://mlsanity.com/paper/2106.07229/) (2021)
- [Stronger Privacy Amplification by Shuffling for R'enyi and Approximate Differential Privacy](https://mlsanity.com/paper/2208.04591/) (2022)
- [Machine Learning in IoT Security: Current Solutions and Future Challenges](https://mlsanity.com/paper/1904.05735/) (2019)
- [EVA: An Encrypted Vector Arithmetic Language and Compiler for Efficient Homomorphic Computation](https://mlsanity.com/paper/1912.11951/) (2019)
- [Orion: A Fully Homomorphic Encryption Framework for Deep Learning](https://mlsanity.com/paper/2311.03470/) (2023)
- [Highly Accurate CNN Inference Using Approximate Activation Functions over Homomorphic Encryption](https://mlsanity.com/paper/2009.03727/) (2020)
- [A2Q: Accumulator-Aware Quantization with Guaranteed Overflow Avoidance](https://mlsanity.com/paper/2308.13504/) (2023)
- [A2Q+: Improving Accumulator-Aware Weight Quantization](https://mlsanity.com/paper/2401.10432/) (2024)
- [Tight Differential Privacy Guarantees for the Shuffle Model with k-Randomized Response](https://mlsanity.com/paper/2205.08858/) (2022)

## Abstract (arXiv)

In contemporary cloud-based services, protecting users' sensitive data and ensuring the confidentiality of the server's model are critical. Fully homomorphic encryption (FHE) enables inference directly on encrypted inputs, but its practicality is hindered by expensive bootstrapping and inefficient approximations of non-linear activations. We introduce Safhire, a hybrid inference framework that executes linear layers under encryption on the server while offloading non-linearities to the client in plaintext. This design eliminates bootstrapping, supports exact activations, and significantly reduces computation. To safeguard model confidentiality despite client access to intermediate outputs, Safhire applies randomized shuffling, which obfuscates intermediate values and makes it practically impossible to reconstruct the model. To further reduce latency, Safhire incorporates advanced optimizations such as fast ciphertext packing and partial extraction. Evaluations on multiple standard models and datasets show that Safhire achieves 1.5X - 10.5X lower inference latency than Orion, a state-of-the-art baseline, with manageable communication overhead and comparable accuracy, thereby establishing the practicality of hybrid FHE inference.