# Multimodal Unlearnable Examples: Protecting Data against Multimodal Contrastive Learning

arXiv: https://arxiv.org/abs/2407.16307 · HTML: https://mlsanity.com/paper/2407.16307/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Jul 2024.

## Built on by (most cited first, 21 of 21)

- [A Survey on Mechanistic Interpretability for Multi-Modal Foundation Models](https://mlsanity.com/paper/2502.17516/) (2025)
- [MMUnlearner: Reformulating Multimodal Machine Unlearning in the Era of Multimodal Large Language Models](https://mlsanity.com/paper/2502.11051/) (2025)
- [MTL-UE: Learning to Learn Nothing for Multi-Task Learning](https://mlsanity.com/paper/2505.05279/) (2025)
- [Gibberish is All You Need for Membership Inference Detection in Contrastive Language-Audio Pretraining](https://mlsanity.com/paper/2410.18371/) (2024)
- [Multimodal Unlearning Across Vision, Language, Video, and Audio: Survey of Methods, Datasets, and Benchmarks](https://mlsanity.com/paper/2607.07907/) (2026)
- [A Survey on Unlearnable Data](https://mlsanity.com/paper/2503.23536/) (2025)
- [T2UE: Generating Unlearnable Examples from Text Descriptions](https://mlsanity.com/paper/2508.03091/) (2025)
- [Dual-Stage Reweighted MoE for Long-Tailed Egocentric Mistake Detection](https://mlsanity.com/paper/2509.12990/) (2025)
- [Full spectrum Unlearnable Examples via Spectral Equalization](https://mlsanity.com/paper/2606.26719/) (2026)
- [Why Do Unlearnable Examples Work: A Novel Perspective of Mutual Information](https://mlsanity.com/paper/2603.03725/) (2026)
- [To See is Not to Learn: Protecting Multimodal Data from Unauthorized Fine-Tuning of Large Vision-Language Model](https://mlsanity.com/paper/2605.14291/) (2026)
- [Adversarial Attacks for Good: A Survey of Proactive Protection across the Visual Content Lifecycle](https://mlsanity.com/paper/2608.04314/) (2026)
- [PersGuard: Preventing Malicious Personalization in Text-to-Image Diffusion Models via Model Backdoors](https://mlsanity.com/paper/2502.16167/) (2025)
- [Multimodal Backdoor Attack on VLMs for Autonomous Driving via Graffiti and Cross-Lingual Triggers](https://mlsanity.com/paper/2604.04630/) (2026)
- [Learning to Generate Cross-Task Unexploitable Examples](https://mlsanity.com/paper/2512.13416/) (2025)
- [Membership Inference for Contrastive Pre-training Models with Text-only PII Queries](https://mlsanity.com/paper/2603.14222/) (2026)
- [Unveiling the Backdoor Mechanism Hidden Behind Catastrophic Overfitting in Fast Adversarial Training](https://mlsanity.com/paper/2604.24350/) (2026)
- [SoK: Unlearnability and Unlearning for Model Dememorization](https://mlsanity.com/paper/2605.11592/) (2026)
- [SoK: A Comprehensive Analysis of the Current Status of Neural Tangent Generalization Attacks with Research Directions](https://mlsanity.com/paper/2605.12792/) (2026)
- [TextCloak: Thwarting Unauthorized LLM Exploitation via RL-Driven Unlearnable Text](https://mlsanity.com/paper/2607.28862/) (2026)
- [Beyond Visual Evidence: Revealing and Mitigating Relational Privacy Leakage in Document MLLMs](https://mlsanity.com/paper/2608.12911/) (2026)

## Abstract (arXiv)

Multimodal contrastive learning (MCL) has shown remarkable advances in zero-shot classification by learning from millions of image-caption pairs crawled from the Internet. However, this reliance poses privacy risks, as hackers may unauthorizedly exploit image-text data for model training, potentially including personal and privacy-sensitive information. Recent works propose generating unlearnable examples by adding imperceptible perturbations to training images to build shortcuts for protection. However, they are designed for unimodal classification, which remains largely unexplored in MCL. We first explore this context by evaluating the performance of existing methods on image-caption pairs, and they do not generalize effectively to multimodal data and exhibit limited impact to build shortcuts due to the lack of labels and the dispersion of pairs in MCL. In this paper, we propose Multi-step Error Minimization (MEM), a novel optimization process for generating multimodal unlearnable examples. It extends the Error-Minimization (EM) framework to optimize both image noise and an additional text trigger, thereby enlarging the optimized space and effectively misleading the model to learn the shortcut between the noise features and the text trigger. Specifically, we adopt projected gradient descent to solve the noise minimization problem and use HotFlip to approximate the gradient and replace words to find the optimal text trigger. Extensive experiments demonstrate the effectiveness of MEM, with post-protection retrieval results nearly half of random guessing, and its high transferability across different models. Our code is available on the https://github.com/thinwayliu/Multimodal-Unlearnable-Examples