# Obsidian: Cooperative State-Space Exploration for Performant Inference on Secure ML Accelerators

arXiv: https://arxiv.org/abs/2409.02817 · HTML: https://mlsanity.com/paper/2409.02817/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 66 references that are in the map.
- First posted on arXiv 4 Sep 2024.

## Builds on (most cited first, 25 of 25)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Identity Mappings in Deep Residual Networks](https://mlsanity.com/paper/1603.05027/) (2016)
- [Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation](https://mlsanity.com/paper/1609.08144/) (2016)
- [In-Datacenter Performance Analysis of a Tensor Processing Unit](https://mlsanity.com/paper/1704.04760/) (2017)
- [Deep Speech: Scaling up end-to-end speech recognition](https://mlsanity.com/paper/1412.5567/) (2014)
- [Deep Learning Recommendation Model for Personalization and Recommendation Systems](https://mlsanity.com/paper/1906.00091/) (2019)
- [EIE: Efficient Inference Engine on Compressed Deep Neural Network](https://mlsanity.com/paper/1602.01528/) (2016)
- [Learning to Optimize Tensor Programs](https://mlsanity.com/paper/1805.08166/) (2018)
- [MLIR: A Compiler Infrastructure for the End of Moore's Law](https://mlsanity.com/paper/2002.11054/) (2020)
- [Black-Box Attacks on Sequential Recommenders via Data-Free Model Extraction](https://mlsanity.com/paper/2109.01165/) (2021)
- [PREMA: A Predictive Multi-task Scheduling Algorithm For Preemptible Neural Processing Units](https://mlsanity.com/paper/1909.04548/) (2019)
- [Mind Mappings: Enabling Efficient Algorithm-Accelerator Mapping Space Search](https://mlsanity.com/paper/2103.01489/) (2021)
- [CoSA: Scheduling by Constrained Optimization for Spatial Accelerators](https://mlsanity.com/paper/2105.01898/) (2021)
- [MAGMA: An Optimization Framework for Mapping Multiple DNNs on Multiple Accelerator Cores](https://mlsanity.com/paper/2104.13997/) (2021)
- [GuardNN: Secure Accelerator Architecture for Privacy-Preserving Deep Learning](https://mlsanity.com/paper/2008.11632/) (2020)
- [A Hardware-Software Blueprint for Flexible Deep Learning Specialization](https://mlsanity.com/paper/1807.04188/) (2018)
- [DiGamma: Domain-aware Genetic Algorithm for HW-Mapping Co-optimization for DNN Accelerators](https://mlsanity.com/paper/2201.11220/) (2022)
- [MGX: Near-Zero Overhead Memory Protection for Data-Intensive Accelerators](https://mlsanity.com/paper/2004.09679/) (2020)
- [ShEF: Shielded Enclaves for Cloud FPGAs](https://mlsanity.com/paper/2103.03500/) (2021)
- [An Energy-Efficient Reconfigurable DTLS Cryptographic Engine for Securing Internet-of-Things Applications](https://mlsanity.com/paper/1907.04455/) (2019)
- [SESAME: Software defined Enclaves to Secure Inference Accelerators with Multi-tenant Execution](https://mlsanity.com/paper/2007.06751/) (2020)
- [Bandwidth Utilization Side-Channel on ML Inference Accelerators](https://mlsanity.com/paper/2110.07157/) (2021)
- [Keystone: An Open Framework for Architecting TEEs](https://mlsanity.com/paper/1907.10119/) (2019)

## Built on by (most cited first, 1 of 1)

- [SoK: A Systems Perspective on Compound AI Threats and Countermeasures](https://mlsanity.com/paper/2411.13459/) (2024)

## Abstract (arXiv)

Trusted execution environments (TEEs) for machine learning accelerators are indispensable in secure and efficient ML inference. Optimizing workloads through state-space exploration for the accelerator architectures improves performance and energy consumption. However, such explorations are expensive and slow due to the large search space. Current research has to use fast analytical models that forego critical hardware details and cross-layer opportunities unique to the hardware security primitives. While cycle-accurate models can theoretically reach better designs, their high runtime cost restricts them to a smaller state space. We present Obsidian, an optimization framework for finding the optimal mapping from ML kernels to a secure ML accelerator. Obsidian addresses the above challenge by exploring the state space using analytical and cycle-accurate models cooperatively. The two main exploration components include: (1) A secure accelerator analytical model, that includes the effect of secure hardware while traversing the large mapping state space and produce the best m model mappings; (2) A compiler profiling step on a cycle-accurate model, that captures runtime bottlenecks to further improve execution runtime, energy and resource utilization and find the optimal model mapping. We compare our results to a baseline secure accelerator, comprising of the state-of-the-art security schemes obtained from guardnn [ 33 ] and sesame [11]. The analytical model reduces the inference latency by 20.5% for a cloud and 8.4% for an edge deployment with an energy improvement of 24% and 19% respectively. The cycle-accurate model, further reduces the latency by 9.1% for a cloud and 12.2% for an edge with an energy improvement of 13.8% and 13.1%.