# Good-Enough LLM Obfuscation (GELO)

arXiv: https://arxiv.org/abs/2603.05035 · HTML: https://mlsanity.com/paper/2603.05035/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Early exit and edge inference, one of 12 directions in Efficient LLM inference and serving.
- Builds on 11 of its 23 references that are in the map.
- First posted on arXiv 5 Mar 2026.

## Placed in

- Efficient LLM inference and serving / [Early exit and edge inference](https://mlsanity.com/inference/early-exit-and-edge-inference/)

## Builds on (most cited first, 11 of 11)

- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distances](https://mlsanity.com/paper/1306.0895/) (2013)
- [How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddings](https://mlsanity.com/paper/1909.00512/) (2019)
- [ShadowNet: A Secure and Efficient On-device Model Inference System for Convolutional Neural Networks](https://mlsanity.com/paper/2011.05905/) (2020)
- [Shadow in the Cache: Unveiling and Mitigating Privacy Risks of KV-cache in LLM Inference](https://mlsanity.com/paper/2508.09442/) (2025)
- [Modeling Shared Responses in Neuroimaging Studies through MultiView ICA](https://mlsanity.com/paper/2006.06635/) (2020)
- [PermLLM: Private Inference of Large Language Models within 3 Seconds under WAN](https://mlsanity.com/paper/2405.18744/) (2024)
- [A First Look At Efficient And Secure On-Device LLM Inference Against KV Leakage](https://mlsanity.com/paper/2409.04040/) (2024)
- [Secure Transformer Inference Protocol](https://mlsanity.com/paper/2312.00025/) (2023)
- [LeftoverLocals: Listening to LLM Responses Through Leaked GPU Local Memory](https://mlsanity.com/paper/2401.16603/) (2024)

## Abstract (arXiv)

Large Language Models (LLMs) are increasingly served on shared accelerators where an adversary with read access to device memory can observe KV caches and hidden states, threatening prompt privacy for open-source models. Cryptographic protections such as MPC and FHE offer strong guarantees but remain one to two orders of magnitude too slow for interactive inference, while static obfuscation schemes break under multi-run statistical attacks once the model is known. We present GELO (Good-Enough LLM Obfuscation), a lightweight protocol for privacy-preserving inference that limits information leakage from untrusted accelerator observations by hiding hidden states with fresh, per-batch invertible mixing. For each offloaded projection, the TEE samples a random matrix A, forms U = AH, offloads U and weights W to the accelerator, and then applies A-1 on return, so that A-1((AH)W ) = HW and outputs are unchanged. Because mixing is never reused across batches, the attacker faces only a single-batch blind source separation problem. We analyse information leakage and introduce two practical defences: (i) non-orthogonal mixing to mask Gram matrices, and (ii) orthogonal mixing augmented with a small fraction of high-energy "shield" vectors that pollute higher-order statistics. On Llama-2 7B, GELO preserves float32 outputs exactly, closely matches low-precision baselines, and shows about 20--30% compute-side overhead in a controlled offload microbenchmark; an unoptimized remote prototype is dominated by transport overhead, motivating deeper serving-engine integration. GELO resists ICA/BSS and anchor-assisted attacks; a 60M-parameter transformer-based unmixing attack also fails under strong mixing and shielding.