# Securing Transformer-based AI Execution via Unified TEEs and Crypto-protected Accelerators

arXiv: https://arxiv.org/abs/2507.03278 · HTML: https://mlsanity.com/paper/2507.03278/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 57 references that are in the map.
- First posted on arXiv 4 Jul 2025.

## Builds on (most cited first, 25 of 25)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Pointer Sentinel Mixture Models](https://mlsanity.com/paper/1609.07843/) (2016)
- [Learning to Communicate with Deep Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1605.06676/) (2016)
- [Post-Training Quantization for Vision Transformer](https://mlsanity.com/paper/2106.14156/) (2021)
- [I-BERT: Integer-only BERT Quantization](https://mlsanity.com/paper/2101.01321/) (2021)
- [Slalom: Fast, Verifiable and Private Execution of Neural Networks in Trusted Hardware](https://mlsanity.com/paper/1806.03287/) (2018)
- [SHE: A Fast and Accurate Deep Neural Network for Encrypted Data](https://mlsanity.com/paper/1906.00148/) (2019)
- [MLCapsule: Guarded Offline Deployment of Machine Learning as a Service](https://mlsanity.com/paper/1808.00590/) (2018)
- [No Privacy Left Outside: On the (In-)Security of TEE-Shielded DNN Partition for On-Device ML](https://mlsanity.com/paper/2310.07152/) (2023)
- [Primer: Fast Private Transformer Inference on Encrypted Data](https://mlsanity.com/paper/2303.13679/) (2023)
- [NNSplitter: An Active Defense Solution for DNN Model via Automated Weight Obfuscation](https://mlsanity.com/paper/2305.00097/) (2023)
- [DarKnight: An Accelerated Framework for Privacy and Integrity Preserving Deep Learning Using Trusted Hardware](https://mlsanity.com/paper/2207.00083/) (2022)
- [CipherPrune: Efficient and Scalable Private Transformer Inference](https://mlsanity.com/paper/2502.16782/) (2025)
- [MirrorNet: A TEE-Friendly Framework for Secure On-device DNN Inference](https://mlsanity.com/paper/2311.09489/) (2023)
- [TensorSCONE: A Secure TensorFlow Framework using Intel SGX](https://mlsanity.com/paper/1902.04413/) (2019)
- [A Survey of Microarchitectural Side-channel Vulnerabilities, Attacks and Defenses in Cryptography](https://mlsanity.com/paper/2103.14244/) (2021)
- [BoostCom: Towards Efficient Universal Fully Homomorphic Encryption by Boosting the Word-wise Comparisons](https://mlsanity.com/paper/2407.07308/) (2024)
- [Trinity: A General Purpose FHE Accelerator](https://mlsanity.com/paper/2410.13405/) (2024)
- [DR.SGX: Hardening SGX Enclaves against Cache Attacks with Data Location Randomization](https://mlsanity.com/paper/1709.09917/) (2017)
- [Tempo: Confidentiality Preservation in Cloud-Based Neural Network Training](https://mlsanity.com/paper/2401.11531/) (2024)

## Built on by (most cited first, 2 of 2)

- [Communication-Efficient Verifiable Attention for LLM Inference](https://mlsanity.com/paper/2606.16352/) (2026)
- [SpliTEE: Fast and Private LLM Inference by Coupling GPU-Assisted Trusted Execution Environments with Differential Privacy](https://mlsanity.com/paper/2609.15039/) (2026)

## Abstract (arXiv)

Recent advances in Transformer models, e.g., large language models (LLMs), have brought tremendous breakthroughs in various artificial intelligence (AI) tasks, leading to their wide applications in many security-critical domains. Due to their unprecedented scale and prohibitively high development cost, these models have become highly valuable intellectual property for AI stakeholders and are increasingly deployed via machine learning as a service (MLaaS). However, MLaaS often runs on untrusted cloud infrastructure, exposing data and models to potential breaches. Mainstream protection mechanisms leverage trusted execution environments (TEEs) where confidentiality and integrity for secretive data are shielded using hardware-based encryption and integrity checking. Unfortunately, running model inference entirely within TEEs is subject to non-trivial slowdown, which is further exacerbated in LLMs due to the substantial computation and memory footprint involved. Recent studies reveal that the hybrid TEE-based scheme offloading partial model inference operations to the untrusted accelerators (e.g., GPU) is a promising solution. However, prior offloading schemes fail to ensure dual protection of data and model in Transformer inference, as they cannot securely offload critical operations, i.e., Attention and SoftMax, forcing these computations to remain confined within TEEs. To address these challenges, we propose TwinShield, a framework enabling secure Transformer inference in heterogeneous TEE and accelerator systems with dual protection for both model and data. TwinShield offloads ~87% of computation to GPUs and delivers 4.0x - 6.1x speedups over previous approaches across various Transformer models.