# "Adversarial Examples" for Proof-of-Learning

arXiv: https://arxiv.org/abs/2108.09454 · HTML: https://mlsanity.com/paper/2108.09454/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 47 references that are in the map.
- First posted on arXiv 21 Aug 2021.

## Builds on (most cited first, 24 of 24)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Advances and Open Problems in Federated Learning](https://mlsanity.com/paper/1912.04977/) (2019)
- [Federated Learning: Challenges, Methods, and Future Directions](https://mlsanity.com/paper/1908.07873/) (2019)
- [Deep Leakage from Gradients](https://mlsanity.com/paper/1906.08935/) (2019)
- [Towards Federated Learning at Scale: System Design](https://mlsanity.com/paper/1902.01046/) (2019)
- [Distillation as a Defense to Adversarial Perturbations against Deep Neural Networks](https://mlsanity.com/paper/1511.04508/) (2015)
- [Stealing Machine Learning Models via Prediction APIs](https://mlsanity.com/paper/1609.02943/) (2016)
- [One pixel attack for fooling deep neural networks](https://mlsanity.com/paper/1710.08864/) (2017)
- [Knockoff Nets: Stealing Functionality of Black-Box Models](https://mlsanity.com/paper/1812.02766/) (2018)
- [cuDNN: Efficient Primitives for Deep Learning](https://mlsanity.com/paper/1410.0759/) (2014)
- [Improving the Robustness of Deep Neural Networks via Stability Training](https://mlsanity.com/paper/1604.04326/) (2016)
- [Stealing Hyperparameters in Machine Learning](https://mlsanity.com/paper/1802.05351/) (2018)
- [Proof-of-Learning: Definitions and Practice](https://mlsanity.com/paper/2103.05633/) (2021)
- [NO Need to Worry about Adversarial Examples in Object Detection in Autonomous Vehicles](https://mlsanity.com/paper/1707.03501/) (2017)
- [Unadversarial Examples: Designing Objects for Robust Vision](https://mlsanity.com/paper/2012.12235/) (2020)
- [Adversarial Examples that Fool Detectors](https://mlsanity.com/paper/1712.02494/) (2017)
- [Enhancing Robustness of Machine Learning Systems via Data Transformations](https://mlsanity.com/paper/1704.02654/) (2017)
- [Invisible Mask: Practical Attacks on Face Recognition with Infrared](https://mlsanity.com/paper/1803.04683/) (2018)
- [Foveation-based Mechanisms Alleviate Adversarial Examples](https://mlsanity.com/paper/1511.06292/) (2015)
- [Learning Adversary-Resistant Deep Neural Networks](https://mlsanity.com/paper/1612.01401/) (2016)

## Built on by (most cited first, 20 of 20)

- [Machine Unlearning: Solutions and Challenges](https://mlsanity.com/paper/2308.07061/) (2023)
- [SA-Attack: Improving Adversarial Transferability of Vision-Language Pre-training Models via Self-Augmentation](https://mlsanity.com/paper/2312.04913/) (2023)
- [Tools for Verifying Neural Models' Training Data](https://mlsanity.com/paper/2307.00682/) (2023)
- [What does it take to catch a Chinchilla? Verifying Rules on Large-Scale Neural Network Training via Compute Monitoring](https://mlsanity.com/paper/2303.11341/) (2023)
- [zkDL: Efficient Zero-Knowledge Proofs of Deep Learning Training](https://mlsanity.com/paper/2307.16273/) (2023)
- [False Claims against Model Ownership Resolution](https://mlsanity.com/paper/2304.06607/) (2023)
- [Machine Learning with Confidential Computing: A Systematization of Knowledge](https://mlsanity.com/paper/2208.10134/) (2022)
- [Proof of Unlearning: Definitions and Instantiation](https://mlsanity.com/paper/2210.11334/) (2022)
- [IronForge: An Open, Secure, Fair, Decentralized Federated Learning](https://mlsanity.com/paper/2301.04006/) (2023)
- [Proof-of-Learning is Currently More Broken Than You Think](https://mlsanity.com/paper/2208.03567/) (2022)
- [On the Robustness of Dataset Inference](https://mlsanity.com/paper/2210.13631/) (2022)
- [Attesting Distributional Properties of Training Data for Machine Learning](https://mlsanity.com/paper/2308.09552/) (2023)
- [Can Membership Inferencing be Refuted?](https://mlsanity.com/paper/2303.03648/) (2023)
- [Proof-of-Learning with Incentive Security](https://mlsanity.com/paper/2404.09005/) (2024)
- [Towards Understanding and Enhancing Security of Proof-of-Training for DNN Model Ownership Verification](https://mlsanity.com/paper/2410.04397/) (2024)
- [GradEscape: A Gradient-Based Evader Against AI-Generated Text Detectors](https://mlsanity.com/paper/2506.08188/) (2025)
- [VerifiableFL: Verifiable Claims for Federated Learning using Exclaves](https://mlsanity.com/paper/2412.10537/) (2024)
- [A2-DIDM: Privacy-preserving Accumulator-enabled Auditing for Distributed Identity of DNN Model](https://mlsanity.com/paper/2405.04108/) (2024)
- [PoLO: Proof-of-Learning and Proof-of-Ownership at Once with Chained Watermarking](https://mlsanity.com/paper/2505.12296/) (2025)
- [SEDULity: A Proof-of-Learning Framework for Distributed and Secure Blockchains with Efficient Useful Work](https://mlsanity.com/paper/2512.13666/) (2025)

## Abstract (arXiv)

In S&P '21, Jia et al. proposed a new concept/mechanism named proof-of-learning (PoL), which allows a prover to demonstrate ownership of a machine learning model by proving integrity of the training procedure. It guarantees that an adversary cannot construct a valid proof with less cost (in both computation and storage) than that made by the prover in generating the proof. A PoL proof includes a set of intermediate models recorded during training, together with the corresponding data points used to obtain each recorded model. Jia et al. claimed that an adversary merely knowing the final model and training dataset cannot efficiently find a set of intermediate models with correct data points. In this paper, however, we show that PoL is vulnerable to ``adversarial examples''! Specifically, in a similar way as optimizing an adversarial example, we could make an arbitrarily-chosen data point ``generate'' a given model, hence efficiently generating intermediate models with correct data points. We demonstrate, both theoretically and empirically, that we are able to generate a valid proof with significantly less cost than generating a proof by the prover.