# Activation Functions Considered Harmful: Recovering Neural Network Weights through Controlled Channels

arXiv: https://arxiv.org/abs/2503.19142 · HTML: https://mlsanity.com/paper/2503.19142/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 63 references that are in the map.
- First posted on arXiv 24 Mar 2025.

## Builds on (most cited first, 20 of 20)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [GLU Variants Improve Transformer](https://mlsanity.com/paper/2002.05202/) (2020)
- [Stealing Machine Learning Models via Prediction APIs](https://mlsanity.com/paper/1609.02943/) (2016)
- [Cache Telepathy: Leveraging Shared Resource Attacks to Learn DNN Architectures](https://mlsanity.com/paper/1808.04761/) (2018)
- [Stealing Neural Networks via Timing Side Channels](https://mlsanity.com/paper/1812.11720/) (2018)
- [Security Analysis of Deep Neural Networks Operating in the Presence of Cache Side-Channel Attacks](https://mlsanity.com/paper/1810.03487/) (2018)
- [Polynomial Time Cryptanalytic Extraction of Neural Network Models](https://mlsanity.com/paper/2310.08708/) (2023)
- [Software Grand Exposure: SGX Cache Attacks Are Practical](https://mlsanity.com/paper/1702.07521/) (2017)
- [Leaky Cauldron on the Dark Land: Understanding Memory Side-Channel Hazards in SGX](https://mlsanity.com/paper/1705.07289/) (2017)
- [A Survey of Published Attacks on Intel SGX](https://mlsanity.com/paper/2006.13598/) (2020)
- [Decompiling x86 Deep Neural Network Executables](https://mlsanity.com/paper/2210.01075/) (2022)
- [CoreGuard: Safeguarding Foundational Capabilities of LLMs Against Model Stealing in Edge Deployment](https://mlsanity.com/paper/2410.13903/) (2024)
- [SEV-Step: A Single-Stepping Framework for AMD-SEV](https://mlsanity.com/paper/2307.14757/) (2023)
- [DR.SGX: Hardening SGX Enclaves against Cache Attacks with Data Location Randomization](https://mlsanity.com/paper/1709.09917/) (2017)
- [On (the Lack of) Code Confidentiality in Trusted Execution Environments](https://mlsanity.com/paper/2212.07899/) (2022)
- [CopyCat: Controlled Instruction-Level Attacks on Enclaves](https://mlsanity.com/paper/2002.08437/) (2020)
- [Unintended Memorization and Timing Attacks in Named Entity Recognition Models](https://mlsanity.com/paper/2211.02245/) (2022)
- [D'ej`a Vu: Side-Channel Analysis of Mozilla's NSS](https://mlsanity.com/paper/2008.06004/) (2020)

## Built on by (most cited first, 2 of 2)

- [TrEEStealer: Stealing Decision Trees via Enclave Side Channels](https://mlsanity.com/paper/2604.18716/) (2026)
- [SparSEEty: Extracting Tokens from Sparsity-Exploiting LLM Serving Systems via Deterministic Side Channels](https://mlsanity.com/paper/2608.02995/) (2026)

## Abstract (arXiv)

With high-stakes machine learning applications increasingly moving to untrusted end-user or cloud environments, safeguarding pre-trained model parameters becomes essential for protecting intellectual property and user privacy. Recent advancements in hardware-isolated enclaves, notably Intel SGX, hold the promise to secure the internal state of machine learning applications even against compromised operating systems. However, we show that privileged software adversaries can exploit input-dependent memory access patterns in common neural network activation functions to extract secret weights and biases from an SGX enclave. Our attack leverages the SGX-Step framework to obtain a noise-free, instruction-granular page-access trace. In a case study of an 11-input regression network using the Tensorflow Microlite library, we demonstrate complete recovery of all first-layer weights and biases, as well as partial recovery of parameters from deeper layers under specific conditions. Our novel attack technique requires only 20 queries per input per weight to obtain all first-layer weights and biases with an average absolute error of less than 1%, improving over prior model stealing attacks. Additionally, a broader ecosystem analysis reveals the widespread use of activation functions with input-dependent memory access patterns in popular machine learning frameworks (either directly or via underlying math libraries). Our findings highlight the limitations of deploying confidential models in SGX enclaves and emphasise the need for stricter side-channel validation of machine learning implementations, akin to the vetting efforts applied to secure cryptographic libraries.