# A Review of Confidentiality Threats Against Embedded Neural Network Models

arXiv: https://arxiv.org/abs/2105.01401 · HTML: https://mlsanity.com/paper/2105.01401/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 30 references that are in the map.
- First posted on arXiv 4 May 2021.

## Builds on (most cited first, 21 of 21)

- [Towards Evaluating the Robustness of Neural Networks](https://mlsanity.com/paper/1608.04644/) (2016)
- [Membership Inference Attacks against Machine Learning Models](https://mlsanity.com/paper/1610.05820/) (2016)
- [The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks](https://mlsanity.com/paper/1802.08232/) (2018)
- [One pixel attack for fooling deep neural networks](https://mlsanity.com/paper/1710.08864/) (2017)
- [ML-Leaks: Model and Data Independent Membership Inference Attacks and Defenses on Machine Learning Models](https://mlsanity.com/paper/1806.01246/) (2018)
- [Label-Only Membership Inference Attacks](https://mlsanity.com/paper/2007.14321/) (2020)
- [Systematic Evaluation of Privacy Risks of Machine Learning Models](https://mlsanity.com/paper/2003.10595/) (2020)
- [White-box vs Black-box: Bayes Optimal Strategies for Membership Inference](https://mlsanity.com/paper/1908.11229/) (2019)
- [High Accuracy and High Fidelity Extraction of Neural Networks](https://mlsanity.com/paper/1909.01838/) (2019)
- [MemGuard: Defending against Black-Box Membership Inference Attacks via Adversarial Examples](https://mlsanity.com/paper/1909.10594/) (2019)
- [Machine Learning with Membership Privacy using Adversarial Regularization](https://mlsanity.com/paper/1807.05852/) (2018)
- [Privacy Risks of Securing Machine Learning Models against Adversarial Examples](https://mlsanity.com/paper/1905.10291/) (2019)
- [Cryptanalytic Extraction of Neural Network Models](https://mlsanity.com/paper/2003.04884/) (2020)
- [Open DNN Box by Power Side-Channel Attack](https://mlsanity.com/paper/1907.10406/) (2019)
- [I Know What You See: Power Side-Channel Attack on Convolutional Neural Network Accelerators](https://mlsanity.com/paper/1803.05847/) (2018)
- [SNIFF: Reverse Engineering of Neural Networks with Fault Attacks](https://mlsanity.com/paper/2002.11021/) (2020)
- [Membership Inference Attacks and Defenses in Classification Models](https://mlsanity.com/paper/2002.12062/) (2020)
- [MaskedNet: The First Hardware Inference Engine Aiming Power Side-Channel Protection](https://mlsanity.com/paper/1910.13063/) (2019)
- [Data and Model Dependencies of Membership Inference Attack](https://mlsanity.com/paper/2002.06856/) (2020)
- [Leaky Nets: Recovering Embedded Neural Network Models and Inputs through Simple Power and Timing Side-Channels -- Attacks and Defenses](https://mlsanity.com/paper/2103.14739/) (2021)
- [Use the Spear as a Shield: A Novel Adversarial Example based Privacy-Preserving Technique against Membership Inference Attacks](https://mlsanity.com/paper/2011.13696/) (2020)

## Built on by (most cited first, 4 of 4)

- [A Practical Introduction to Side-Channel Extraction of Deep Neural Network Parameters](https://mlsanity.com/paper/2211.05590/) (2022)
- [Physical Side-Channel Attacks on Embedded Neural Networks: A Survey](https://mlsanity.com/paper/2110.11290/) (2021)
- [QuantAttack: Exploiting Dynamic Quantization to Attack Vision Transformers](https://mlsanity.com/paper/2312.02220/) (2023)
- [DeSparsify: Adversarial Attack Against Token Sparsification Mechanisms in Vision Transformers](https://mlsanity.com/paper/2402.02554/) (2024)

## Abstract (arXiv)

Utilization of Machine Learning (ML) algorithms, especially Deep Neural Network (DNN) models, becomes a widely accepted standard in many domains more particularly IoT-based systems. DNN models reach impressive performances in several sensitive fields such as medical diagnosis, smart transport or security threat detection, and represent a valuable piece of Intellectual Property. Over the last few years, a major trend is the large-scale deployment of models in a wide variety of devices. However, this migration to embedded systems is slowed down because of the broad spectrum of attacks threatening the integrity, confidentiality and availability of embedded models. In this review, we cover the landscape of attacks targeting the confidentiality of embedded DNN models that may have a major impact on critical IoT systems, with a particular focus on model extraction and data leakage. We highlight the fact that Side-Channel Analysis (SCA) is a relatively unexplored bias by which model's confidentiality can be compromised. Input data, architecture or parameters of a model can be extracted from power or electromagnetic observations, testifying a real need from a security point of view.