# Cryptanalytic Extraction of Neural Network Models

arXiv: https://arxiv.org/abs/2003.04884 · HTML: https://mlsanity.com/paper/2003.04884/

## Key facts

- Cited by 68 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Mar 2020.

## Built on by (most cited first, 30 of 68)

- [Reconstructing Training Data from Trained Neural Networks](https://mlsanity.com/paper/2206.07758/) (2022)
- [I Know What You Trained Last Summer: A Survey on Stealing Machine Learning Models and Defences](https://mlsanity.com/paper/2206.08451/) (2022)
- [SoK: How Robust is Image Classification Deep Neural Network Watermarking? (Extended Version)](https://mlsanity.com/paper/2108.04974/) (2021)
- [Fingerprinting Deep Neural Networks Globally via Universal Adversarial Perturbations](https://mlsanity.com/paper/2202.08602/) (2022)
- [Model Stealing Attacks Against Inductive Graph Neural Networks](https://mlsanity.com/paper/2112.08331/) (2021)
- [Copy, Right? A Testing Framework for Copyright Protection of Deep Learning Models](https://mlsanity.com/paper/2112.05588/) (2021)
- [DeepSteal: Advanced Model Extractions Leveraging Efficient Weight Stealing in Memories](https://mlsanity.com/paper/2111.04625/) (2021)
- [Hidden symmetries of ReLU networks](https://mlsanity.com/paper/2306.06179/) (2023)
- [TnT Attacks! Universal Naturalistic Adversarial Patches Against Deep Neural Network Systems](https://mlsanity.com/paper/2111.09999/) (2021)
- [Parameter identifiability of a deep feedforward ReLU neural network](https://mlsanity.com/paper/2112.12982/) (2021)
- [StolenEncoder: Stealing Pre-trained Encoders in Self-supervised Learning](https://mlsanity.com/paper/2201.05889/) (2022)
- [Privacy Backdoors: Stealing Data with Corrupted Pretrained Models](https://mlsanity.com/paper/2404.00473/) (2024)
- [An Embedding of ReLU Networks and an Analysis of their Identifiability](https://mlsanity.com/paper/2107.09370/) (2021)
- [False Claims against Model Ownership Resolution](https://mlsanity.com/paper/2304.06607/) (2023)
- [Polynomial Time Cryptanalytic Extraction of Neural Network Models](https://mlsanity.com/paper/2310.08708/) (2023)
- [AI Risk Management Should Incorporate Both Safety and Security](https://mlsanity.com/paper/2405.19524/) (2024)
- [Teacher Model Fingerprinting Attacks Against Transfer Learning](https://mlsanity.com/paper/2106.12478/) (2021)
- [GrOVe: Ownership Verification of Graph Neural Networks using Embeddings](https://mlsanity.com/paper/2304.08566/) (2023)
- [Local Identifiability of Deep ReLU Neural Networks: the Theory](https://mlsanity.com/paper/2206.07424/) (2022)
- [Conflicting Interactions Among Protection Mechanisms for Machine Learning Models](https://mlsanity.com/paper/2207.01991/) (2022)
- [MIST: Defending Against Membership Inference Attacks Through Membership-Invariant Subspace Training](https://mlsanity.com/paper/2311.00919/) (2023)
- [An Exact Poly-Time Membership-Queries Algorithm for Extraction a three-Layer ReLU Network](https://mlsanity.com/paper/2105.09673/) (2021)
- [Split HE: Fast Secure Inference Combining Split Learning and Homomorphic Encryption](https://mlsanity.com/paper/2202.13351/) (2022)
- [SoK: Machine Learning Governance](https://mlsanity.com/paper/2109.10870/) (2021)
- [A Practical Introduction to Side-Channel Extraction of Deep Neural Network Parameters](https://mlsanity.com/paper/2211.05590/) (2022)
- [Polynomial Time Cryptanalytic Extraction of Deep Neural Networks in the Hard-Label Setting](https://mlsanity.com/paper/2410.05750/) (2024)
- [Model Extraction and Defenses on Generative Adversarial Networks](https://mlsanity.com/paper/2101.02069/) (2021)
- [Responsible and Regulatory Conform Machine Learning for Medicine: A Survey of Challenges and Solutions](https://mlsanity.com/paper/2107.09546/) (2021)
- [Beyond Slow Signs in High-fidelity Model Extraction](https://mlsanity.com/paper/2406.10011/) (2024)
- [10 Security and Privacy Problems in Large Foundation Models](https://mlsanity.com/paper/2110.15444/) (2021)

## Abstract (arXiv)

We argue that the machine learning problem of model extraction is actually a cryptanalytic problem in disguise, and should be studied as such. Given oracle access to a neural network, we introduce a differential attack that can efficiently steal the parameters of the remote model up to floating point precision. Our attack relies on the fact that ReLU neural networks are piecewise linear functions, and thus queries at the critical points reveal information about the model parameters. We evaluate our attack on multiple neural network models and extract models that are 2^20 times more precise and require 100x fewer queries than prior work. For example, we extract a 100,000 parameter neural network trained on the MNIST digit recognition task with 2^21.5 queries in under an hour, such that the extracted model agrees with the oracle on all inputs up to a worst-case error of 2^-25, or a model with 4,000 parameters in 2^18.5 queries with worst-case error of 2^-40.4. Code is available at https://github.com/google-research/cryptanalytic-model-extraction.