# PRISM: Sensitivity-Aware PolynoMial PRuning for EffIcient Neural Network Encryption

arXiv: https://arxiv.org/abs/2607.18342 · HTML: https://mlsanity.com/paper/2607.18342/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 15 of its 34 references that are in the map.
- First posted on arXiv 20 Jul 2026.

## Builds on (most cited first, 15 of 15)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Pruning Filters for Efficient ConvNets](https://mlsanity.com/paper/1608.08710/) (2016)
- [Channel Pruning for Accelerating Very Deep Neural Networks](https://mlsanity.com/paper/1707.06168/) (2017)
- [Gazelle: A Low Latency Framework for Secure Neural Network Inference](https://mlsanity.com/paper/1801.05507/) (2018)
- [DeepReDuce: ReLU Reduction for Fast Private Inference](https://mlsanity.com/paper/2103.01396/) (2021)
- [CryptoNAS: Private Inference on a ReLU Budget](https://mlsanity.com/paper/2006.08733/) (2020)
- [AutoFHE: Automated Adaption of CNNs for Efficient Evaluation over FHE](https://mlsanity.com/paper/2310.08012/) (2023)
- [AESPA: Accuracy Preserving Low-degree Polynomial Activation for Fast Private Inference](https://mlsanity.com/paper/2201.06699/) (2022)
- [Highly Accurate CNN Inference Using Approximate Activation Functions over Homomorphic Encryption](https://mlsanity.com/paper/2009.03727/) (2020)
- [HyPHEN: A Hybrid Packing Method and Optimizations for Homomorphic Encryption-Based Neural Networks](https://mlsanity.com/paper/2302.02407/) (2023)
- [PrivCirNet: Efficient Private Inference via Block Circulant Transformation](https://mlsanity.com/paper/2405.14569/) (2024)
- [Efficient Pruning for Machine Learning Under Homomorphic Encryption](https://mlsanity.com/paper/2207.03384/) (2022)
- [MOFHEI: Model Optimizing Framework for Fast and Efficient Homomorphically Encrypted Neural Network Inference](https://mlsanity.com/paper/2412.07954/) (2024)
- [Reliability Analysis of Fully Homomorphic Encryption Systems Under Memory Faults](https://mlsanity.com/paper/2509.20686/) (2025)
- [On the Vulnerability of FHE Computation to Silent Data Corruption](https://mlsanity.com/paper/2603.23253/) (2026)

## Abstract (arXiv)

Structured pruning is essential for making neural network inference feasible under homomorphic encryption (HE), yet its impact on model reliability has remained unexplored. This paper presents a systematic reliability characterization of pruned CKKS-encrypted neural networks and introduces Polynomial-Sensitivity-Aware Pruning (PSAP), a structured pruning method that is inherently reliability-aware. PSAP scores filters jointly by weight magnitude, polynomial activation sensitivity, and rotation cost, which concentrates pruning in fault-tolerant regions. Across two architectures, two datasets, two numerical representations, and five bit-error rates (40 full-model and 108 per-layer experiments), PSAP-pruned models limit catastrophic (>10 pp accuracy drop) layers to at most two versus 5--14 for magnitude-pruned baselines, reducing worst-case vulnerability by up to 29 times under int32 bit-flip injection. Direct CKKS encrypted fault injection indicates a safe operating boundary near BER~ 10-5, supporting int32 injection as a conservative reliability proxy. The fault-critical structural layers account for only 1.1% of parameters, enabling selective hardening at minimal overhead. These reliability gains are obtained alongside competitive efficiency: PSAP reduces Halevi--Shoup rotations by up to 45.2% on ResNet-32, and an adaptive mixed-degree allocation scheme lowers multiplicative depth from 66 to 56 levels, enabling leveled inference without bootstrapping.