mlsanityData to 14 Oct 2026

CLEAR: Character Unlearning in Textual and Visual Modalities

arXiv 2410.18057 ↗First posted 23 Oct 2024cs.CV cs.CLcited by 39

On the map: LLM safety and security → Machine unlearning

Abstract (arXiv)

Machine Unlearning (MU) is critical for removing private or hazardous information from deep learning models. While MU has advanced significantly in unimodal (text or vision) settings, multimodal unlearning (MMU) remains underexplored due to the lack of open benchmarks for evaluating cross-modal data removal. To address this gap, we introduce CLEAR, the first open-source benchmark designed specifically for MMU. CLEAR contains 200 fictitious individuals and 3,700 images linked with corresponding question-answer pairs, enabling a thorough evaluation across modalities. We conduct a comprehensive analysis of 11 MU methods (e.g., SCRUB, gradient ascent, DPO) across four evaluation sets, demonstrating that jointly unlearning both modalities outperforms single-modality approaches. The dataset is available at https://huggingface.co/datasets/therem/CLEAR

Key facts. Cited by 39 papers in the map (data through 14 Oct 2026). Placed in Machine unlearning, one of 12 directions in LLM safety and security. Builds on 43 of its 61 references that are in the map. First posted on arXiv 23 Oct 2024.

Builds on

43 of 61 references.

Closest ancestors · 13

Same direction, newest first

  1. 2024MUSE: Machine Unlearning Six-Way Evaluation for Language Models8 Jul
  2. 2024Negative Preference Optimization: From Catastrophic Collapse to Effective Unlearning8 Apr
  3. 2024Inexact Unlearning Needs More Careful Evaluations to Avoid a False Sense of Privacy2 Mar
  4. 2024Towards Safer Large Language Models through Machine Unlearning15 Feb
  5. 2024TOFU: A Task of Fictitious Unlearning for LLMs11 Jan
  6. 2023Unlearn What You Want to Forget: Efficient Unlearning for LLMs31 Oct
  7. 2023Who's Harry Potter? Approximate Unlearning in LLMs3 Oct
  8. 2023Model Sparsity Can Simplify Machine Unlearning11 Apr
  9. 2022Knowledge Unlearning for Mitigating Privacy Risks in Language Models4 Oct
  10. 2022Quark: Controllable Text Generation with Reinforced Unlearning26 May
  11. 2022Continual Learning and Private Unlearning24 Mar
  12. 2021Fast Yet Effective Machine Unlearning17 Nov
1 more
  1. 2021Remember What You Want to Forget: Algorithms for Machine Unlearning4 Mar

Other references · 30

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  5. 2022Training language models to follow instructions with human feedbackcited by 15,971
  6. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  7. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  8. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  9. 2014Deep Learning Face Attributes in the Wildcited by 3,693
  10. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,588
  11. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,975
  12. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,516
18 more
  1. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,281
  2. 2021Membership Inference Attacks From First Principlescited by 878
  3. 2023BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Datasetcited by 860
  4. 2019Interpreting the Latent Space of GANs for Semantic Face Editingcited by 695
  5. 2019Eternal Sunshine of the Spotless Net: Selective Forgetting in Deep Networkscited by 618
  6. 2024The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearningcited by 509
  7. 2014Learning Face Representation from Scratchcited by 453
  8. 2023FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Promptscited by 398
  9. 2020StyleSpace Analysis: Disentangled Controls for StyleGAN Image Generationcited by 332
  10. 2023PhotoMaker: Customizing Realistic Human Photos via Stacked ID Embeddingcited by 328
  11. 2020Descent-to-Delete: Gradient-Based Methods for Machine Unlearningcited by 311
  12. 2023Jailbreak in pieces: Compositional Adversarial Attacks on Multi-Modal Language Modelscited by 273
  13. 2024JailBreakV: A Benchmark for Assessing the Robustness of MultiModal Large Language Models against Jailbreak Attackscited by 230
  14. 2023Composing Parameter-Efficient Modules with Arithmetic Operationscited by 156
  15. 2023Identifying and Mitigating the Security Risks of Generative AIcited by 91
  16. 2024UnlearnCanvas: Stylized Image Dataset for Enhanced Machine Unlearning Evaluation in Diffusion Modelscited by 50
  17. 2024Cross-Modal Safety Alignment: Is textual unlearning all you need?cited by 25
  18. 2023QuickDrop: Efficient Federated Unlearning by Integrated Dataset Distillationcited by 11

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 12; 2026: 27.

39 papers.

Sort:

By area:

  1. 2025A Comprehensive Survey of Machine Unlearning Techniques for Large Language Modelscited by 34
  2. 2025MMUnlearner: Reformulating Multimodal Machine Unlearning in the Era of Multimodal Large Language Modelscited by 30
  3. 2025Safety Mirage: How Spurious Correlations Undermine VLM Safety Fine-Tuning and Can Be Mitigated by Machine Unlearningcited by 16
  4. 2025SAUCE: Selective Concept Unlearning in Vision-Language Models with Sparse Autoencoderscited by 13
  5. 2025MLLMEraser: Achieving Test-Time Unlearning in Multimodal Large Language Models through Activation Steeringcited by 10
  6. 2025OFFSIDE: Benchmarking Unlearning Misinformation in Multimodal Large Language Modelscited by 8
  7. 2025Cross-Modal Unlearning via Influential Neuron Path Editing in Multimodal Large Language Modelscited by 8
  8. 2026Multimodal Unlearning Across Vision, Language, Video, and Audio: Survey of Methods, Datasets, and Benchmarkscited by 7
  9. 2026Metis: Memory Foundation Modelcited by 7
  10. 2025PEBench: A Fictitious Dataset to Benchmark Machine Unlearning for Multimodal Large Language Modelscited by 6
  11. 2025Towards Benign Memory Forgetting for Selective Multimodal Large Language Model Unlearningcited by 6
  12. 2025SUA: Stealthy Multimodal Large Language Model Unlearning Attackcited by 4
27 more
  1. 2025Generative Model Unlearning: A Survey through Target Events, Unlearning Operators, and Evaluation Protocolscited by 4
  2. 2026Visual-Guided Key-Token Regularization for Multimodal Large Language Model Unlearningcited by 3
  3. 2026Before Forgetting, Learn to Remember: Revisiting Foundational Learning Failures in LVLM Unlearning Benchmarkscited by 3
  4. 2026Null Space Constrained Contrastive Visual Forgetting for MLLM Unlearningcited by 3
  5. 2026Visual-Noise Guided In-Context Distillation for Multimodal Large Language Model Unlearningcited by 3
  6. 2025AUVIC: Adversarial Unlearning of Visual Concepts for Multi-modal Large Language Modelscited by 2
  7. 2026Erase Persona, Forget Lore: Benchmarking Multimodal Copyright Unlearning in Large Vision Language Modelscited by 2
  8. 2026Anatomy of Unlearning: The Dual Impact of Fact Salience and Model Fine-Tuningcited by 1
  9. 2026Suppression Is Not Forgetting: Residual Recoverability in Visual Concept Unlearning for VLMscited by 1
  10. 2026ICU-Bench:Benchmarking Continual Unlearning in Multimodal Large Language Modelscited by 1
  11. 2026PPU-Bench:Real World Benchmark for Personalized Partial Unlearning in Vision Language Modelscited by 1
  12. 2026MemLeak: Diagnosing Information Leaks in Multimodal Agent Memorycited by 1
  13. 2026Does Forgetting Transfer Across Modalities? A Real-World Benchmark for Cross-Modal Knowledge Unlearning Evaluationcited by 1
  14. 2026Defending Wearable VLMs Against Private Attribute Inferencecited by 1
  15. 2026Where Identity Lives: Localized, Retain-Free Identity Unlearning in Multimodal Large Language Modelscited by 1
  16. 2026PPE-Bench: A Benchmark for Evaluating MLLM Unlearning under Private-Public Entanglementcited by 0
  17. 2026POPS: Recovering Unlearned Multi-Modality Knowledge in MLLMs with Prompt-Optimized Parameter Shakingcited by 0
  18. 2026One Modality to Forget Them All: Enhancing Cross-Modal Unlearning in Vision-Language Modelscited by 0
  19. 2026Unlearning Under Imbalance: Benchmarking Fairness in Multimodal LLM Unlearningcited by 0
  20. 2026Toward Fine-Grained Forgetting:Attribute Unlearning for Multimodal Large Language Modelscited by 0
  21. 2026A Model Merging Approach for Continual MLLM Unlearningcited by 0
  22. 2026PRMU: A Corpus-Free Benchmark for Person-Centric Knowledge Unlearning in Multimodal Large Language Modelscited by 0
  23. 2026Confess What You Know: Forget-Set Misalignment with Model Knowledge in LLM Unlearningcited by 0
  24. 2026SIEVE: Selective attention-value Suppression for Vision-Language Models Unlearningcited by 0
  25. 2026SAGE: Efficient Protected Vision-Language Inference on Smart Glassescited by 0
  26. 2026CUSE: Fine-Grained Multimodal Unlearning with Counterfactual Guidance and On-Policy Self-Distillationcited by 0
  27. 2026Are We Really Measuring Forgetting? Pitfalls in Evaluating Machine Unlearning for Multimodal LLMscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Single Image Unlearning: Efficient Machine Unlearning in Multimodal Large Language Models
  2. 2024MU-Bench: A Multitask Multimodal Benchmark for Machine Unlearning
  3. 2025Unlearning Sensitive Information in Multimodal LLMs: Benchmark and Attack-Defense Evaluation
  4. 2024Both Text and Images Leaked! A Systematic Analysis of Data Contamination in Multimodal LLM
  5. 2025SafeEraser: Enhancing Safety in Multimodal Large Language Models through Multimodal Machine Unlearning
  6. 2025Preserving Cross-Modal Stability for Visual Unlearning in Multimodal Scenarios
  7. 2024Protecting Privacy in Multimodal Large Language Models with MLLMU-Bench
  8. 2026SALMUBench: A Benchmark for Sensitive Association-Level Multimodal Unlearning