# AEGIS: Scaling Long-Sequence Homomorphic Encrypted Transformer Inference via Hybrid Parallelism on Multi-GPU Systems

arXiv: https://arxiv.org/abs/2604.03425 · HTML: https://mlsanity.com/paper/2604.03425/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 57 references that are in the map.
- First posted on arXiv 3 Apr 2026.

## Builds on (most cited first, 19 of 19)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity](https://mlsanity.com/paper/2101.03961/) (2021)
- [GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding](https://mlsanity.com/paper/2006.16668/) (2020)
- [Efficient Large-Scale Language Model Training on GPU Clusters Using Megatron-LM](https://mlsanity.com/paper/2104.04473/) (2021)
- [Gazelle: A Low Latency Framework for Secure Neural Network Inference](https://mlsanity.com/paper/1801.05507/) (2018)
- [Online normalizer calculation for softmax](https://mlsanity.com/paper/1805.02867/) (2018)
- [Sequence Parallelism: Long Sequence Training from System Perspective](https://mlsanity.com/paper/2105.13120/) (2021)
- [TenSEAL: A Library for Encrypted Tensor Operations Using Homomorphic Encryption](https://mlsanity.com/paper/2104.03152/) (2021)
- [F1: A Fast and Programmable Accelerator for Fully Homomorphic Encryption (Extended Version)](https://mlsanity.com/paper/2109.05371/) (2021)
- [HeLayers: A Tile Tensors Framework for Large Neural Networks on Encrypted Data](https://mlsanity.com/paper/2011.01805/) (2020)
- [EVA: An Encrypted Vector Arithmetic Language and Compiler for Efficient Homomorphic Computation](https://mlsanity.com/paper/1912.11951/) (2019)
- [Orion: A Fully Homomorphic Encryption Framework for Deep Learning](https://mlsanity.com/paper/2311.03470/) (2023)
- [BTS: An Accelerator for Bootstrappable Fully Homomorphic Encryption](https://mlsanity.com/paper/2112.15479/) (2021)
- [HETAL: Efficient Privacy-preserving Transfer Learning with Homomorphic Encryption](https://mlsanity.com/paper/2403.14111/) (2024)
- [Secure Human Action Recognition by Encrypted Neural Network Inference](https://mlsanity.com/paper/2104.09164/) (2021)
- [CryptoGCN: Fast and Scalable Homomorphically Encrypted Graph Convolutional Network Inference](https://mlsanity.com/paper/2209.11904/) (2022)
- [A Scalable Multi-GPU Framework for Encrypted Large-Model Inference](https://mlsanity.com/paper/2512.11269/) (2025)
- [Porcupine: A Synthesizing Compiler for Vectorized Homomorphic Encryption](https://mlsanity.com/paper/2101.07841/) (2021)
- [FicGCN: Unveiling the Homomorphic Encryption Efficiency from Irregular Graph Convolutional Networks](https://mlsanity.com/paper/2506.10399/) (2025)

## Built on by (most cited first, 2 of 2)

- [Public-Decay Homomorphic State Space Models for Private Sequence Inference](https://mlsanity.com/paper/2605.16647/) (2026)
- [FEnc2: Unifying Data Packing for Efficient Private Inference via Convolution and Architecture-Aware Fragment Encoding](https://mlsanity.com/paper/2606.16359/) (2026)

## Abstract (arXiv)

Fully Homomorphic Encryption (FHE) enables privacy-preserving Transformer inference, but long-sequence encrypted Transformers quickly exceed single-GPU memory capacity because encoded weights are already large and encrypted activations grow rapidly with sequence length. Multi-GPU execution therefore becomes unavoidable, yet scaling remains challenging because communication is jointly induced by application-level aggregation and encryption-level RNS coupling. Existing approaches either synchronize between devices frequently or replicate encrypted tensors across devices, leading to excessive communication and latency. We present AEGIS, an Application-Encryption Guided Inference System for scalable long-sequence encrypted Transformer inference on multi-GPU platforms. AEGIS derives device placement from ciphertext dependencies jointly induced by Transformer dataflow and CKKS polynomial coupling, co-locating modulus-coherent and token-coherent data so that communication is introduced only when application dependencies require it, while reordering polynomial operators to overlap the remaining collectives with computation. On 2048-token inputs, AEGIS reduces inter-GPU communication by up to 57.9% in feed-forward networks and 81.3% in self-attention versus prior state-of-the-art designs. On four GPUs, it achieves up to 96.62% scaling efficiency, 3.86x end-to-end speedup, and 69.1% per-device memory reduction. These results establish coordinated application-encryption parallelism as a practical foundation for scalable homomorphic Transformer inference.