mlsanityData to 14 Oct 2026

Converting Transformers to Polynomial Form for Secure Inference Over Homomorphic Encryption

arXiv 2311.08610 ↗First posted 15 Nov 2023cs.LG cs.CRcited by 19

Not on the map. Why?

Abstract (arXiv)

Designing privacy-preserving deep learning models is a major challenge within the deep learning community. Homomorphic Encryption (HE) has emerged as one of the most promising approaches in this realm, enabling the decoupling of knowledge between the model owner and the data owner. Despite extensive research and application of this technology, primarily in convolutional neural networks, incorporating HE into transformer models has been challenging because of the difficulties in converting these models into a polynomial form. We break new ground by introducing the first polynomial transformer, providing the first demonstration of secure inference over HE with transformers. This includes a transformer architecture tailored for HE, alongside a novel method for converting operators to their polynomial equivalent. This innovation enables us to perform secure inference on LMs with WikiText-103. It also allows us to perform image classification with CIFAR-100 and Tiny-ImageNet. Our models yield results comparable to traditional methods, bridging the performance gap with transformers of similar scale and underscoring the viability of HE for state-of-the-art applications. Finally, we assess the stability of our models and conduct a series of ablations to quantify the contribution of each model component.

Key facts. Cited by 19 papers in the map (data through 14 Oct 2026). Builds on 18 of its 41 references that are in the map. First posted on arXiv 15 Nov 2023.

Builds on

18 of 41 references.

References · 18

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2022Mega: Moving Average Equipped Gated Attentioncited by 179
  4. 2021Primer: Searching for Efficient Transformers for Language Modelingcited by 164
  5. 2021Privacy-Preserving Machine Learning with Fully Homomorphic Encryption for Deep Neural Networkcited by 57
  6. 2022THE-X: Privacy-Preserving Transformer Inference with Homomorphic Encryptioncited by 57
  7. 2017CryptoDL: Deep Neural Networks over Encrypted Datacited by 56
  8. 2020Pi-nets: Deep Polynomial Neural Networkscited by 39
  9. 2021Precise Approximation of Convolutional Neural Networks for Homomorphically Encrypted Datacited by 27
  10. 2020HeLayers: A Tile Tensors Framework for Large Neural Networks on Encrypted Datacited by 22
  11. 2023LLMs Can Understand Encrypted Prompt: Towards Privacy-Computing Friendly Transformerscited by 22
  12. 2022MPCViT: Searching for Accurate and Efficient MPC-Friendly Vision Transformer with Heterogeneous Attentioncited by 20
6 more
  1. 2023Primer: Fast Private Transformer Inference on Encrypted Datacited by 20
  2. 2023AutoFHE: Automated Adaption of CNNs for Efficient Evaluation over FHEcited by 15
  3. 2022Understanding the Failure of Batch Normalization for Transformers in NLPcited by 12
  4. 2021A methodology for training homomorphicencryption friendly neural networkscited by 10
  5. 2022Privacy-Preserving Text Classification on BERT Embeddings with Homomorphic Encryptioncited by 9
  6. 2019Privacy preserving Neural Network Inference on Encrypted Data with GPUscited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 7; 2025: 6; 2026: 6.

19 papers.

Sort:

By area:

  1. 2024The Hidden Attention of Mamba Modelscited by 91
  2. 2024Encryption-Friendly LLM Architecturecited by 17
  3. 2024SLIP: Securing LLMs IP Using Weights Decompositioncited by 14
  4. 2025CipherPrune: Efficient and Scalable Private Transformer Inferencecited by 10
  5. 2024Power-Softmax: Towards Secure LLM Inference over Encrypted Datacited by 7
  6. 2025LLM Access Shield: Domain-Specific LLM Framework for Privacy Policy Compliancecited by 6
  7. 2025Efficient Decoding Methods for Language Models on Encrypted Datacited by 5
  8. 2025On the Expressivity of Selective State-Space Layers: A Multivariate Polynomial Approachcited by 4
  9. 2024ReLU's Revival: On the Entropic Overload in Normalization-Free Large Language Modelscited by 3
  10. 2024AERO: Entropy-Guided Framework for Private LLM Inferencecited by 3
  11. 2025A Training Framework for Optimal and Stable Training of Polynomial Neural Networkscited by 3
  12. 2025An Efficient Private GPT Never Autoregressively Decodescited by 2
7 more
  1. 2026ATLAS: Automated Approximation of Transformers for Efficient Homomorphic Inference in One Hourcited by 2
  2. 2026FuseFSS: Efficient Secure LLM Inference with Function Secret Sharingcited by 1
  3. 2026The Art of Mixology: Mixup-based Obfuscation for Privacy-Preserving Split Learning in Large Language Modelscited by 1
  4. 2024Efficient Privacy-Preserving KAN Inference Using Homomorphic Encryptioncited by 0
  5. 2026Gecko: Fast Private Inference via Secure Public Encoder Offloadingcited by 0
  6. 2026HEAT: Faster Fully Homomorphic Inference via Approximations-Weights Co-Adaptationcited by 0
  7. 2026DegreeSpar: Structured Degree Sparsity for Efficient Secure Transformer Inferencecited by 0