mlsanityData to 13 Oct 2026

SACE: Concept Erasure at the Semantic Singularity in Visual Autoregressive Models

arXiv 2606.15819 ↗First posted 14 Jun 2026cs.CV cs.AIcited by 0

On the map: LLM alignment / RLHF / preference optimisation → Diffusion and multimodal model alignment

also in 1 more
Abstract (arXiv)

The rapid progress of visual autoregressive (VAR) models has unlocked a transformative frontier for high-fidelity text-to-image synthesis, while heightening concerns over the safety alignment of generated content. Naive application of existing erasure techniques to VAR models causes catastrophic semantic collapse and visual artifacts, since they are predominantly designed for the homogeneous denoising steps of diffusion models. To address this foundational challenge, we first propose the Semantic Singularity Axiom, which posits that any target semantic concept embedded within a prompt is definitively locked at Scale-0. Then rigorously validate this axiom through our proposed Incremental Semantic Saliency Analysis (ISSA),which also enable the community to transparently inspect the coarse-to-fine semantic injection process. Guided by this insight, we introduce the first scale-aware concept erasure framework (SACE) for VAR models. By strictly confining interventions to the first scale, our approach couples an Entropy-Regularized Erasure Objective to prevent high-entropy sampling degeneration, alongside a restorative preservation loss to safely anchor the integrity of entangled benign priors. Extensive experiments demonstrate that our method achieves surgical concept erasure performance across various domains with minimal training overhead, timely and elegently resolute the critical safety vulnerabilities inherent in emerging VAR architectures. Code is available at: https://github.com/limerenceysy/SACE}{https://github.com/limerenceysy/SACE.

Key facts. No citing papers in the map yet (data through 13 Oct 2026). Placed in Diffusion and multimodal model alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 44 of its 54 references that are in the map. First posted on arXiv 14 Jun 2026.

Builds on

44 of 54 references.

References · 44

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,952
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,777
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,550
  4. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,028
  5. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,564
  6. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,339
  7. 2022Scalable Diffusion Models with Transformerscited by 5,973
  8. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,237
  9. 2024Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Predictioncited by 1,012
  10. 2016Pixel Recurrent Neural Networkscited by 815
  11. 2025Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scalingcited by 799
  12. 2023Erasing Concepts from Diffusion Modelscited by 535
32 more
  1. 2022Safe Latent Diffusion: Mitigating Inappropriate Degeneration in Diffusion Modelscited by 509
  2. 2023Ablating Concepts in Text-to-Image Diffusion Modelscited by 316
  3. 2024MACE: Mass Concept Erasure in Diffusion Modelscited by 303
  4. 2023Forget-Me-Not: Learning to Forget in Text-to-Image Diffusion Modelscited by 301
  5. 2024Infinity: Scaling Bitwise AutoRegressive Modeling for High-Resolution Image Synthesiscited by 300
  6. 2023To Generate or Not? Safety-Driven Unlearned Diffusion Models Are Still Easy To Generate Unsafe Images ... For Nowcited by 210
  7. 2023Selective Amnesia: A Continual Learning Approach to Forgetting in Deep Generative Modelscited by 177
  8. 2023One-Dimensional Adapter to Rule Them All: Concepts, Diffusion Models and Erasing Applicationscited by 169
  9. 2024Defensive Unlearning with Adversarial Training for Robust Concept Erasure in Diffusion Modelscited by 168
  10. 2023Editing Implicit Assumptions in Text-to-Image Diffusion Modelscited by 129
  11. 2023Receler: Reliable Concept Erasing of Text-to-Image Diffusion Models via Lightweight Eraserscited by 120
  12. 2024Unlearning Concepts in Diffusion Model via Concept Domain Correction and Concept Preserving Gradientcited by 69
  13. 2024SafeGen: Mitigating Sexually Explicit Content Generation in Text-to-Image Modelscited by 65
  14. 2024R.A.C.E.: Robust Adversarial Concept Erasure for Secure Text-to-Image Diffusion Modelcited by 58
  15. 2024ConceptPrune: Concept Editing in Diffusion Models via Skilled Neuron Pruningcited by 54
  16. 2024Erasing Undesirable Concepts in Diffusion Models with Adversarial Preservationcited by 50
  17. 2025InfinityStar: Unified Spacetime AutoRegressive Modeling for Visual Generationcited by 43
  18. 2023Towards Safe Self-Distillation of Internet-Scale Text-to-Image Diffusion Modelscited by 41
  19. 2023All but One: Surgical Concept Erasing with Model Preservation in Text-to-Image Diffusion Modelscited by 33
  20. 2023Degeneration-Tuning: Using Scrambled Grid shield Unwanted Concepts from Stable Diffusioncited by 27
  21. 2024Separable Multi-Concept Erasure from Diffusion Modelscited by 27
  22. 2023Implicit Concept Removal of Diffusion Modelscited by 23
  23. 2024Pruning for Robust Concept Erasing in Diffusion Modelscited by 22
  24. 2025Concept Pinpoint Eraser for Text-to-image Diffusion Models via Residual Attention Gatecited by 16
  25. 2024RealEra: Semantic-level Concept Erasure via Neighbor-Concept Miningcited by 15
  26. 2025Closing the Safety Gap: Surgical Concept Erasure in Visual Autoregressive Modelscited by 8
  27. 2025Semantic Surgery: Zero-Shot Concept Erasure in Diffusion Modelscited by 8
  28. 2024Continuous Concepts Removal in Text-to-image Diffusion Modelscited by 6
  29. 2025EAR: Erasing Concepts from Unified Autoregressive Modelscited by 4
  30. 2025UMAMI: Unifying Masked Autoregressive Models and Deterministic Rendering for View Synthesiscited by 4
  31. 2026Conditioned Activation Transport for T2I Safety Steeringcited by 3
  32. 2026KidsNanny: A Two-Stage Multimodal Content Moderation Pipeline Integrating Visual Classification, Object Detection, OCR, and Contextual Reasoning for Child Safetycited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Erasing Without Collateral Damage: Precise Concept Removal in Diffusion Models
  2. 2025Erased or Dormant? Rethinking Concept Erasure Through Reversibility
  3. 2026GRACE: Adaptive Concept Erasure with Geometry-Guided Retention in Diffusion Models
  4. 2026Localized Concept Erasure in Text-to-Image Diffusion Models via High-Level Representation Misdirection
  5. 2025Autoregressive Semantic Visual Reconstruction Helps VLMs Understand Better
  6. 2026CleanVideo: Adaptive Concept Erasure for Text-to-Video Diffusion Models
  7. 2026CoreUnlearn: Rethinking Concept Unlearning through Disentangled Component-Level Erasure in Text-guided Diffusion Models
  8. 2025A Comprehensive Survey on Concept Erasure in Text-to-Image Diffusion Models