mlsanityData to 14 Oct 2026

SineProject: Machine Unlearning for Stable Vision Language Alignment

arXiv 2511.18444 ↗First posted 23 Nov 2025cs.CVcited by 3

On the map: Multimodal LLMs / vision-language models → MLLM jailbreaks and adversarial robustness

also in 1 more
Abstract (arXiv)

Multimodal Large Language Models (MLLMs) increasingly need to forget specific knowledge such as unsafe or private information without requiring full retraining. However, existing unlearning methods often disrupt vision language alignment, causing models to reject both harmful and benign queries. We trace this failure to the projector network during unlearning, its Jacobian becomes severely illconditioned, leading to unstable optimization and drift in cross modal embeddings. We introduce SineProject, a simple method that augments the frozen projector with sinusoidally modulated trainable parameters, improving the Jacobian's spectral conditioning and stabilizing alignment throughout unlearning. Across standard safety and privacy unlearning benchmarks using LLaVA v1.5 7B and 13B, SineProject reduces benign query refusals while achieving complete forgetting of targeted information, yielding state of the art forget retain trade offs with negligible computational overhead.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 45 of its 59 references that are in the map. First posted on arXiv 23 Nov 2025.

Builds on

45 of 59 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025SafeEraser: Enhancing Safety in Multimodal Large Language Models through Multimodal Machine Unlearning18 Feb
  2. 2024Protecting Privacy in Multimodal Large Language Models with MLLMU-Bench29 Oct
  3. 2024Single Image Unlearning: Efficient Machine Unlearning in Multimodal Large Language Models21 May

Other references · 42

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  3. 2023Visual Instruction Tuningcited by 7,869
  4. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  5. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
  6. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,322
  7. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  8. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,876
  9. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,621
  10. 2020Implicit Neural Representations with Periodic Activation Functionscited by 2,238
  11. 2021Align before Fuse: Vision and Language Representation Learning with Momentum Distillationcited by 1,619
  12. 2025SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Featurescited by 1,188
30 more
  1. 2019Machine Unlearningcited by 1,061
  2. 2023VILA: On Pre-training for Visual Language Modelscited by 692
  3. 2024LLaVA-NeXT-Interleave: Tackling Multi-image, Video, and 3D in Large Multimodal Modelscited by 584
  4. 2021LiT: Zero-Shot Transfer with Locked-image text Tuningcited by 577
  5. 2024TOFU: A Task of Fictitious Unlearning for LLMscited by 533
  6. 2019Does Learning Require Memorization? A Short Tale about a Long Tailcited by 460
  7. 2022A Survey of Machine Unlearningcited by 312
  8. 2024MUSE: Machine Unlearning Six-Way Evaluation for Language Modelscited by 261
  9. 2023Unlearn What You Want to Forget: Efficient Unlearning for LLMscited by 215
  10. 2017Spectral Norm Regularization for Improving the Generalizability of Deep Learningcited by 182
  11. 2024Efficient multi-prompt evaluation of LLMscited by 73
  12. 2022Backdoor Defense with Machine Unlearningcited by 65
  13. 2025Unlearning Isn't Deletion: Investigating Reversibility of Machine Unlearning in LLMscited by 42
  14. 2025RandLoRA: Full-rank parameter-efficient fine-tuning of large modelscited by 35
  15. 2024Efficient Learning With Sine-Activated Low-rank Matricescited by 22
  16. 2024Deciphering Cross-Modal Alignment in Large Vision-Language Models with Modality Integration Ratecited by 20
  17. 2025A Survey on Efficient Vision-Language Modelscited by 19
  18. 2024MU-Bench: A Multitask Multimodal Benchmark for Machine Unlearningcited by 17
  19. 2024Zero-Shot Class Unlearning in CLIP with Synthetic Samplescited by 17
  20. 2021How to Train Your Wide Neural Network Without Backprop: An Input-Weight Alignment Perspectivecited by 13
  21. 2025Always Skip Attentioncited by 13
  22. 2023Combating Representation Learning Disparity with Geometric Harmonizationcited by 11
  23. 2025Towards Higher Effective Rank in Parameter-efficient Fine-tuning using Khatri--Rao Productcited by 10
  24. 2024Weight Conditioning for Smooth Optimization of Neural Networkscited by 9
  25. 2024Rethinking Attention: Polynomial Alternatives to Softmax in Transformerscited by 8
  26. 2022On skip connections and normalisation layers in deep optimisationcited by 7
  27. 2025Structured Initialization for Vision Transformerscited by 6
  28. 2025EVOREFUSE: Evolutionary Prompt Optimization for Evaluation and Mitigation of LLM Over-Refusal to Pseudo-Malicious Instructionscited by 6
  29. 2025Leaner Transformers: More Heads, Less Depthcited by 5
  30. 2024Leveraging MLLM Embeddings and Attribute Smoothing for Compositional Zero-Shot Learningcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026Beyond Objective Expressivity: Geometry Preservation in Multimodal Contrastive Learningcited by 1
  2. 2026AIM: Anchor Identity Features, Then Match for Multimodal Large Language Model Unlearningcited by 1
  3. 2026Open-MMUnlearning: Unifying Methods and Evaluation for MLLM Unlearningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026One Modality to Forget Them All: Enhancing Cross-Modal Unlearning in Vision-Language Models
  2. 2024Assessing and Learning Alignment of Unimodal Vision and Language Models
  3. 2026Stochastic Meta-Unlearning: Bridging Language Backbone and Multimodal Unlearning
  4. 2026On the Robustness of Machine Unlearning for Vision-Language Models
  5. 2026CATA: Continual Machine Unlearning via Conflict-Averse Task Arithmetic
  6. 2025Unlearning the Noisy Correspondence Makes CLIP More Robust
  7. 2026Visual-Noise Guided In-Context Distillation for Multimodal Large Language Model Unlearning
  8. 2026PPU-Bench:Real World Benchmark for Personalized Partial Unlearning in Vision Language Models