# VerIDeep: Verifying Integrity of Deep Neural Networks through Sensitive-Sample Fingerprinting

arXiv: https://arxiv.org/abs/1808.03277 · HTML: https://mlsanity.com/paper/1808.03277/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Aug 2018.

## Built on by (most cited first, 8 of 8)

- [pvCNN: Privacy-Preserving and Verifiable Convolutional Neural Network Testing](https://mlsanity.com/paper/2201.09186/) (2022)
- [An Auditing Test To Detect Behavioral Shift in Language Models](https://mlsanity.com/paper/2410.19406/) (2024)
- [Robust and Large-Payload DNN Watermarking via Fixed, Distribution-Optimized, Weights](https://mlsanity.com/paper/2208.10973/) (2022)
- [DeepiSign: Invisible Fragile Watermark to Protect the Integrityand Authenticity of CNN](https://mlsanity.com/paper/2101.04319/) (2021)
- [Never trust, always verify : a roadmap for Trustworthy AI?](https://mlsanity.com/paper/2206.11981/) (2022)
- [Robust and Lossless Fingerprinting of Deep Neural Networks via Pooled Membership Inference](https://mlsanity.com/paper/2209.04113/) (2022)
- [Neural-BO: A Black-box Optimization Algorithm using Deep Neural Networks](https://mlsanity.com/paper/2303.01682/) (2023)
- [MPC-Friendly Commitments for Publicly Verifiable Covert Security](https://mlsanity.com/paper/2109.07461/) (2021)

## Abstract (arXiv)

Deep learning has become popular, and numerous cloud-based services are provided to help customers develop and deploy deep learning applications. Meanwhile, various attack techniques have also been discovered to stealthily compromise the model's integrity. When a cloud customer deploys a deep learning model in the cloud and serves it to end-users, it is important for him to be able to verify that the deployed model has not been tampered with, and the model's integrity is protected. We propose a new low-cost and self-served methodology for customers to verify that the model deployed in the cloud is intact, while having only black-box access (e.g., via APIs) to the deployed model. Customers can detect arbitrary changes to their deep learning models. Specifically, we define textttSensitive-Sample fingerprints, which are a small set of transformed inputs that make the model outputs sensitive to the model's parameters. Even small weight changes can be clearly reflected in the model outputs, and observed by the customer. Our experiments on different types of model integrity attacks show that we can detect model integrity breaches with high accuracy (>99%) and low overhead (<10 black-box model accesses).