# TensorShield: Safeguarding On-Device Inference by Shielding Critical DNN Tensors with TEE

arXiv: https://arxiv.org/abs/2505.22735 · HTML: https://mlsanity.com/paper/2505.22735/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 May 2025.

## Built on by (most cited first, 1 of 1)

- [SoK: Attack and Defense Landscape of Mobile On-device AI Systems](https://mlsanity.com/paper/2607.00362/) (2026)

## Abstract (arXiv)

To safeguard user data privacy, on-device inference has emerged as a prominent paradigm on mobile and Internet of Things (IoT) devices. This paradigm involves deploying a model provided by a third party on local devices to perform inference tasks. However, it exposes the private model to two primary security threats: model stealing (MS) and membership inference attacks (MIA). To mitigate these risks, existing wisdom deploys models within Trusted Execution Environments (TEEs), which is a secure isolated execution space. Nonetheless, the constrained secure memory capacity in TEEs makes it challenging to achieve full model security with low inference latency. This paper fills the gap with TensorShield, the first efficient on-device inference work that shields partial tensors of the model while still fully defending against MS and MIA. The key enabling techniques in TensorShield include: (i) a novel eXplainable AI (XAI) technique exploits the model's attention transition to assess critical tensors and shields them in TEE to achieve secure inference, and (ii) two meticulous designs with critical feature identification and latency-aware placement to accelerate inference while maintaining security. Extensive evaluations show that TensorShield delivers almost the same security protection as shielding the entire model inside TEE, while being up to 25.35× (avg. 5.85×) faster than the state-of-the-art work, without accuracy loss.