# Static Bootstrap Placement for Encrypted Language Model Decoding

arXiv: https://arxiv.org/abs/2610.04912 · HTML: https://mlsanity.com/paper/2610.04912/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 19 of its 63 references that are in the map.
- First posted on arXiv 4 Oct 2026.

## Placed in

- Synthetic data and self-improvement for LLMs

## Builds on (most cited first, 19 of 19)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Mistral 7B](https://mlsanity.com/paper/2310.06825/) (2023)
- [PUMA: Secure Inference of LLaMA-7B in Five Minutes](https://mlsanity.com/paper/2307.12533/) (2023)
- [Encryption-Friendly LLM Architecture](https://mlsanity.com/paper/2410.02486/) (2024)
- [Orion: A Fully Homomorphic Encryption Framework for Deep Learning](https://mlsanity.com/paper/2311.03470/) (2023)
- [ARK: Fully Homomorphic Encryption Accelerator with Runtime Data Generation and Inter-Operation Key Reuse](https://mlsanity.com/paper/2205.00922/) (2022)
- [Private and Reliable Neural Network Inference](https://mlsanity.com/paper/2210.15614/) (2022)
- [Cheddar: A Swift Fully Homomorphic Encryption Library Designed for GPU Architectures](https://mlsanity.com/paper/2407.13055/) (2024)
- [HEIR: A Universal Compiler for Homomorphic Encryption](https://mlsanity.com/paper/2508.11095/) (2025)
- [Efficient Decoding Methods for Language Models on Encrypted Data](https://mlsanity.com/paper/2509.08383/) (2025)
- [Cachemir: Fully Homomorphic Encrypted Inference of Generative Large Language Model with KV Cache](https://mlsanity.com/paper/2602.11470/) (2026)
- [FIDESlib: A Fully-Fledged Open-Source FHE Library for Efficient CKKS on GPUs](https://mlsanity.com/paper/2507.04775/) (2025)
- [A Scalable Multi-GPU Framework for Encrypted Large-Model Inference](https://mlsanity.com/paper/2512.11269/) (2025)
- [An Efficient Private GPT Never Autoregressively Decodes](https://mlsanity.com/paper/2505.15252/) (2025)
- [HE-LRM: Encrypted Deep Learning Recommendation Models using Fully Homomorphic Encryption](https://mlsanity.com/paper/2506.18150/) (2025)
- [Scaling up FHE-based Privacy-Preserving ML: Higher Throughput, Longer Inputs for LLama-3-8B](https://mlsanity.com/paper/2601.18511/) (2026)
- [Encrypted control for networked systems -- An illustrative introduction and current challenges](https://mlsanity.com/paper/2010.00268/) (2020)
- [Traveling Salesman-Based Token Ordering Improves Stability in Homomorphically Encrypted Language Models](https://mlsanity.com/paper/2510.12343/) (2025)
- [ROSETTA: Efficient and Accurate Privacy-Preserving LLM Decoding via Hybrid CKKS/TFHE Evaluation](https://mlsanity.com/paper/2609.16915/) (2026)

## Abstract (arXiv)

Language models increasingly serve prompts that carry private data, and secure inference under homomorphic encryption lets a client outsource the computation without revealing the prompt. Existing secure inference systems run a forward pass without consuming a token under encryption, and generating text with them requires a client round trip at every generated token. Keeping the loop on the server instead requires selecting and consuming a token under encryption, and placing bootstraps for a loop body that grows with the context. We build AR-HE, which runs the whole loop on the server, selects each token under encryption, retrieves its embedding, and writes it back into the encrypted state. The client sends one prompt and remains offline until the output. One rule places every bootstrap in the run, without search, so the bootstrap cost of a token is a formula in the context length that is known before the run starts. The schedule skips work whose result cannot reach the output, packs bootstraps that share an operand, and keeps the keys and values of past positions in an encrypted cache. With every optimization applied, generating a GPT-2 small token costs 544 seconds on one NVIDIA H100, down from 4715 seconds without optimization. The prompt step before it costs 4630 seconds. The cache alone takes a generated step from 11751 bootstraps to 1072. The formula predicts every step we measured, including steps of a model it was not derived from.