# Privacy-Preserving Hierarchical Model-Distributed Inference

arXiv: https://arxiv.org/abs/2407.18353 · HTML: https://mlsanity.com/paper/2407.18353/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 44 references that are in the map.
- First posted on arXiv 25 Jul 2024.

## Builds on (most cited first, 16 of 16)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Gazelle: A Low Latency Framework for Secure Neural Network Inference](https://mlsanity.com/paper/1801.05507/) (2018)
- [CrypTFlow2: Practical 2-Party Secure Inference](https://mlsanity.com/paper/2010.06457/) (2020)
- [FALCON: Honest-Majority Maliciously Secure Framework for Private Deep Learning](https://mlsanity.com/paper/2004.02229/) (2020)
- [Chameleon: A Hybrid Secure Computation Framework for Machine Learning Applications](https://mlsanity.com/paper/1801.03239/) (2018)
- [CrypTFlow: Secure TensorFlow Inference](https://mlsanity.com/paper/1909.07814/) (2019)
- [Low Latency Privacy Preserving Inference](https://mlsanity.com/paper/1812.10659/) (2018)
- [CryptoDL: Deep Neural Networks over Encrypted Data](https://mlsanity.com/paper/1711.05189/) (2017)
- [XONN: XNOR-based Oblivious Deep Neural Network Inference](https://mlsanity.com/paper/1902.07342/) (2019)
- [QUOTIENT: Two-Party Secure Neural Network Training and Prediction](https://mlsanity.com/paper/1907.03372/) (2019)
- [nGraph-HE: A Graph Compiler for Deep Learning on Homomorphically Encrypted Data](https://mlsanity.com/paper/1810.10121/) (2018)
- [Pipeline Parallelism for Inference on Heterogeneous Edge Computing](https://mlsanity.com/paper/2110.14895/) (2021)
- [AutoPrivacy: Automated Layer-wise Parameter Selection for Secure Neural Network Inference](https://mlsanity.com/paper/2006.04219/) (2020)
- [SSNet: A Lightweight Multi-Party Computation Scheme for Practical Privacy-Preserving Machine Learning Service in the Cloud](https://mlsanity.com/paper/2406.02629/) (2024)
- [SECO: Secure Inference With Model Splitting Across Multi-Server Hierarchy](https://mlsanity.com/paper/2404.16232/) (2024)

## Built on by (most cited first, 1 of 1)

- [A Distributed Framework for Privacy-Enhanced Vision Transformers on the Edge](https://mlsanity.com/paper/2512.09309/) (2025)

## Abstract (arXiv)

This paper focuses on designing a privacy-preserving Machine Learning (ML) inference protocol for a hierarchical setup, where clients own/generate data, model owners (cloud servers) have a pre-trained ML model, and edge servers perform ML inference on clients' data using the cloud server's ML model. Our goal is to speed up ML inference while providing privacy to both data and the ML model. Our approach (i) uses model-distributed inference (model parallelization) at the edge servers and (ii) reduces the amount of communication to/from the cloud server. Our privacy-preserving hierarchical model-distributed inference, privateMDI design uses additive secret sharing and linearly homomorphic encryption to handle linear calculations in the ML inference, and garbled circuit and a novel three-party oblivious transfer are used to handle non-linear functions. privateMDI consists of offline and online phases. We designed these phases in a way that most of the data exchange is done in the offline phase while the communication overhead of the online phase is reduced. In particular, there is no communication to/from the cloud server in the online phase, and the amount of communication between the client and edge servers is minimized. The experimental results demonstrate that privateMDI significantly reduces the ML inference time as compared to the baselines.