# What Can We Learn from Unlearnable Datasets?

arXiv: https://arxiv.org/abs/2305.19254 · HTML: https://mlsanity.com/paper/2305.19254/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 38 references that are in the map.
- First posted on arXiv 30 May 2023.

## Builds on (most cited first, 25 of 25)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [Adversarial Examples Are Not Bugs, They Are Features](https://mlsanity.com/paper/1905.02175/) (2019)
- [Robustness May Be at Odds with Accuracy](https://mlsanity.com/paper/1805.12152/) (2018)
- [The Open Images Dataset V4: Unified image classification, object detection, and visual relationship detection at scale](https://mlsanity.com/paper/1811.00982/) (2018)
- [Last Layer Re-Training is Sufficient for Robustness to Spurious Correlations](https://mlsanity.com/paper/2204.02937/) (2022)
- [Large image datasets: A pyrrhic win for computer vision?](https://mlsanity.com/paper/2006.16923/) (2020)
- [Unlearnable Examples: Making Personal Data Unexploitable](https://mlsanity.com/paper/2101.04898/) (2021)
- [Adversarial Examples Make Strong Poisons](https://mlsanity.com/paper/2106.10807/) (2021)
- [Learning to Confuse: Generating Training Time Adversarial Data with Auto-Encoder](https://mlsanity.com/paper/1905.09027/) (2019)
- [Availability Attacks Create Shortcuts](https://mlsanity.com/paper/2111.00898/) (2021)
- [Better Safe Than Sorry: Preventing Delusive Adversaries with Adversarial Training](https://mlsanity.com/paper/2102.04716/) (2021)
- [Autoregressive Perturbations for Data Poisoning](https://mlsanity.com/paper/2206.03693/) (2022)
- [Data Poisoning Won't Save You From Facial Recognition](https://mlsanity.com/paper/2106.14851/) (2021)
- [Preventing Unauthorized Use of Proprietary Data: Poisoning for Secure Dataset Release](https://mlsanity.com/paper/2103.02683/) (2021)
- [Learning the Unlearnable: Adversarial Augmentations Suppress Unlearnable Example Attacks](https://mlsanity.com/paper/2303.15127/) (2023)
- [One-Pixel Shortcut: on the Learning Preference of Deep Neural Networks](https://mlsanity.com/paper/2205.12141/) (2022)
- [The Devil's Advocate: Shattering the Illusion of Unexploitable Data using Diffusion Models](https://mlsanity.com/paper/2303.08500/) (2023)
- [Poisons that are learned faster are more effective](https://mlsanity.com/paper/2204.08615/) (2022)
- [Learnability Lock: Authorized Learnability Control Through Adversarial Invertible Transformations](https://mlsanity.com/paper/2202.03576/) (2022)

## Built on by (most cited first, 21 of 21)

- [Purify Unlearnable Examples via Rate-Constrained Variational Autoencoders](https://mlsanity.com/paper/2405.01460/) (2024)
- [UnSeg: One Universal Unlearnable Example Generator is Enough against All Image Segmentation](https://mlsanity.com/paper/2410.09909/) (2024)
- [Semantic Deep Hiding for Robust Unlearnable Examples](https://mlsanity.com/paper/2406.17349/) (2024)
- [Efficient Availability Attacks against Supervised and Contrastive Learning Simultaneously](https://mlsanity.com/paper/2402.04010/) (2024)
- [ECLIPSE: Expunging Clean-label Indiscriminate Poisons via Sparse Diffusion Purification](https://mlsanity.com/paper/2406.15093/) (2024)
- [Toward Availability Attacks in 3D Point Clouds](https://mlsanity.com/paper/2407.11011/) (2024)
- [Nonlinear Transformations Against Unlearnable Datasets](https://mlsanity.com/paper/2406.02883/) (2024)
- [A Survey on Unlearnable Data](https://mlsanity.com/paper/2503.23536/) (2025)
- [When Priors Backfire: On the Vulnerability of Unlearnable Examples to Pretraining](https://mlsanity.com/paper/2603.04731/) (2026)
- [T2UE: Generating Unlearnable Examples from Text Descriptions](https://mlsanity.com/paper/2508.03091/) (2025)
- [Full spectrum Unlearnable Examples via Spectral Equalization](https://mlsanity.com/paper/2606.26719/) (2026)
- [Why Do Unlearnable Examples Work: A Novel Perspective of Mutual Information](https://mlsanity.com/paper/2603.03725/) (2026)
- [Adversarial Attacks for Good: A Survey of Proactive Protection across the Visual Content Lifecycle](https://mlsanity.com/paper/2608.04314/) (2026)
- [Unlearnable Examples Give a False Sense of Data Privacy: Understanding and Relearning](https://mlsanity.com/paper/2306.02064/) (2023)
- [Be Persistent: Towards a Unified Solution for Mitigating Shortcuts in Deep Learning](https://mlsanity.com/paper/2402.11237/) (2024)
- [Beyond Full Poisoning: Effective Availability Attacks with Partial Perturbation](https://mlsanity.com/paper/2407.02437/) (2024)
- [Perturbation-Induced Linearization: Constructing Unlearnable Data with Solely Linear Classifiers](https://mlsanity.com/paper/2601.19967/) (2026)
- [Transfer-based Adversarial Poisoning Attacks for Online (MIMO-)Deep Receviers](https://mlsanity.com/paper/2409.02430/) (2024)
- [Teleology-Driven Affective Computing: A Causal Framework for Sustained Well-Being](https://mlsanity.com/paper/2502.17172/) (2025)
- [SoK: Unlearnability and Unlearning for Model Dememorization](https://mlsanity.com/paper/2605.11592/) (2026)
- [SoK: A Comprehensive Analysis of the Current Status of Neural Tangent Generalization Attacks with Research Directions](https://mlsanity.com/paper/2605.12792/) (2026)

## Abstract (arXiv)

In an era of widespread web scraping, unlearnable dataset methods have the potential to protect data privacy by preventing deep neural networks from generalizing. But in addition to a number of practical limitations that make their use unlikely, we make a number of findings that call into question their ability to safeguard data. First, it is widely believed that neural networks trained on unlearnable datasets only learn shortcuts, simpler rules that are not useful for generalization. In contrast, we find that networks actually can learn useful features that can be reweighed for high test performance, suggesting that image protection is not assured. Unlearnable datasets are also believed to induce learning shortcuts through linear separability of added perturbations. We provide a counterexample, demonstrating that linear separability of perturbations is not a necessary condition. To emphasize why linearly separable perturbations should not be relied upon, we propose an orthogonal projection attack which allows learning from unlearnable datasets published in ICML 2021 and ICLR 2023. Our proposed attack is significantly less complex than recently proposed techniques.