# Beyond Slow Signs in High-fidelity Model Extraction

arXiv: https://arxiv.org/abs/2406.10011 · HTML: https://mlsanity.com/paper/2406.10011/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 21 references that are in the map.
- First posted on arXiv 14 Jun 2024.

## Builds on (most cited first, 16 of 16)

- [Practical Black-Box Attacks against Machine Learning](https://mlsanity.com/paper/1602.02697/) (2016)
- [Stealing Machine Learning Models via Prediction APIs](https://mlsanity.com/paper/1609.02943/) (2016)
- [Knockoff Nets: Stealing Functionality of Black-Box Models](https://mlsanity.com/paper/1812.02766/) (2018)
- [High Accuracy and High Fidelity Extraction of Neural Networks](https://mlsanity.com/paper/1909.01838/) (2019)
- [Data-Free Model Extraction](https://mlsanity.com/paper/2011.14779/) (2020)
- [Model Reconstruction from Model Explanations](https://mlsanity.com/paper/1807.05185/) (2018)
- [Reverse-Engineering Deep ReLU Networks](https://mlsanity.com/paper/1910.00744/) (2019)
- [I Know What You Trained Last Summer: A Survey on Stealing Machine Learning Models and Defences](https://mlsanity.com/paper/2206.08451/) (2022)
- [Cryptanalytic Extraction of Neural Network Models](https://mlsanity.com/paper/2003.04884/) (2020)
- [Hermes Attack: Steal DNN Models with Lossless Inference Accuracy](https://mlsanity.com/paper/2006.12784/) (2020)
- [Polynomial Time Cryptanalytic Extraction of Neural Network Models](https://mlsanity.com/paper/2310.08708/) (2023)
- [Expand-and-Cluster: Parameter Recovery of Neural Networks](https://mlsanity.com/paper/2304.12794/) (2023)
- [CSI Neural Network: Using Side-channels to Recover Your Artificial Neural Network Information](https://mlsanity.com/paper/1810.09076/) (2018)
- [Beyond Labeling Oracles: What does it mean to steal ML models?](https://mlsanity.com/paper/2310.01959/) (2023)
- [Copycat CNN: Are Random Non-Labeled Data Enough to Steal Knowledge from Black-box Models?](https://mlsanity.com/paper/2101.08717/) (2021)
- [InferNet: Exploiting Aggregate GPU Profiles as Side-Channel for DNN Architecture Inference](https://mlsanity.com/paper/2304.03388/) (2023)

## Built on by (most cited first, 7 of 7)

- [Polynomial Time Cryptanalytic Extraction of Deep Neural Networks in the Hard-Label Setting](https://mlsanity.com/paper/2410.05750/) (2024)
- [Navigating the Deep: End-to-End Extraction on Deep Neural Networks](https://mlsanity.com/paper/2506.17047/) (2025)
- [Is the Hard-Label Cryptanalytic Model Extraction Really Polynomial?](https://mlsanity.com/paper/2510.06692/) (2025)
- [Train to Defend: First Defense Against Cryptanalytic Neural Network Parameter Extraction Attacks](https://mlsanity.com/paper/2509.16546/) (2025)
- [Data Augmentation Techniques to Reverse-Engineer Neural Network Weights from Input-Output Queries](https://mlsanity.com/paper/2511.20312/) (2025)
- [Cryptanalytic Extraction of Isolated Bias-Free GLU Feed-Forward Blocks by Antipodal Separation](https://mlsanity.com/paper/2608.06631/) (2026)
- [End-to-End Hard-Label Cryptanalytic Model Extraction Using Efficient Sign Recovery](https://mlsanity.com/paper/2609.21941/) (2026)

## Abstract (arXiv)

Deep neural networks, costly to train and rich in intellectual property value, are increasingly threatened by model extraction attacks that compromise their confidentiality. Previous attacks have succeeded in reverse-engineering model parameters up to a precision of float64 for models trained on random data with at most three hidden layers using cryptanalytical techniques. However, the process was identified to be very time consuming and not feasible for larger and deeper models trained on standard benchmarks. Our study evaluates the feasibility of parameter extraction methods of Carlini et al. [1] further enhanced by Canales-Mart'inez et al. [2] for models trained on standard benchmarks. We introduce a unified codebase that integrates previous methods and reveal that computational tools can significantly influence performance. We develop further optimisations to the end-to-end attack and improve the efficiency of extracting weight signs by up to 14.8 times compared to former methods through the identification of easier and harder to extract neurons. Contrary to prior assumptions, we identify extraction of weights, not extraction of weight signs, as the critical bottleneck. With our improvements, a 16,721 parameter model with 2 hidden layers trained on MNIST is extracted within only 98 minutes compared to at least 150 minutes previously. Finally, addressing methodological deficiencies observed in previous studies, we propose new ways of robust benchmarking for future model extraction attacks.