# Closing the Safety Gap: Surgical Concept Erasure in Visual Autoregressive Models

arXiv: https://arxiv.org/abs/2509.22400 · HTML: https://mlsanity.com/paper/2509.22400/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Sep 2025.

## Built on by (most cited first, 9 of 9)

- [Differential Vector Erasure: Unified Training-Free Concept Erasure for Flow Matching Models](https://mlsanity.com/paper/2602.01089/) (2026)
- [FlowErase-RL: Rethinking Concept Erasure as Reward Optimization in Flow Matching Models](https://mlsanity.com/paper/2605.19739/) (2026)
- [Obliviate: Erasing Concepts from Autoregressive Image Generation Models](https://mlsanity.com/paper/2606.28643/) (2026)
- [FlowErase-OPD: Multi-Concept Erasure via Anchored On-Policy Distillation in Flow Matching Models](https://mlsanity.com/paper/2608.07620/) (2026)
- [C2P-VAR: Continual and Compositional Personalization in Visual Autoregressive Models](https://mlsanity.com/paper/2605.19750/) (2026)
- [SACE: Concept Erasure at the Semantic Singularity in Visual Autoregressive Models](https://mlsanity.com/paper/2606.15819/) (2026)
- [Concept Removal for Frontier Image Generative Models](https://mlsanity.com/paper/2606.25548/) (2026)
- [LEMUR: Latent Entropy-aware Multimodal Unlearning via Visual-anchored Reasoning Redirection](https://mlsanity.com/paper/2608.11691/) (2026)
- [HyperErase: Scale-Calibrated Hypernetwork for Multi-Concept Erasure in Text-to-Image Models](https://mlsanity.com/paper/2609.31154/) (2026)

## Abstract (arXiv)

The rapid progress of visual autoregressive (VAR) models has brought new opportunities for text-to-image generation, but also heightened safety concerns. Existing concept erasure techniques, primarily designed for diffusion models, fail to generalize to VARs due to their next-scale token prediction paradigm. In this paper, we first propose a novel VAR Erasure framework VARE that enables stable concept erasure in VAR models by leveraging auxiliary visual tokens to reduce fine-tuning intensity. Building upon this, we introduce S-VARE, a novel and effective concept erasure method designed for VAR, which incorporates a filtered cross entropy loss to precisely identify and minimally adjust unsafe visual tokens, along with a preservation loss to maintain semantic fidelity, addressing the issues such as language drift and reduced diversity introduce by na"ive fine-tuning. Extensive experiments demonstrate that our approach achieves surgical concept erasure while preserving generation quality, thereby closing the safety gap in autoregressive text-to-image generation by earlier methods.