mlsanityData to 13 Oct 2026

Conditioned Activation Transport for T2I Safety Steering

arXiv 2603.03163 ↗First posted 3 Mar 2026cs.CV cs.AIcited by 3

On the map: Mechanistic interpretability → Steering and probing for LLM safety

Abstract (arXiv)

Despite their impressive capabilities, current Text-to-Image (T2I) models remain prone to generating unsafe and toxic content. While activation steering offers a promising inference-time intervention, we observe that linear activation steering frequently degrades image quality when applied to benign prompts. To address this trade-off, we first construct SafeSteerDataset, a contrastive dataset containing 2300 safe and unsafe prompt pairs with high cosine similarity. Leveraging this data, we propose Conditioned Activation Transport (CAT), a framework that employs a geometry-based conditioning mechanism and nonlinear transport maps. By conditioning transport maps to activate only within unsafe activation regions, we minimize interference with benign queries. We validate our approach on two state-of-the-art architectures: Z-Image and Infinity. Experiments demonstrate that CAT generalizes effectively across these backbones, significantly reducing Attack Success Rate while maintaining image fidelity compared to unsteered generations. Warning: This paper contains potentially offensive text and images.

Key facts. Cited by 3 papers in the map (data through 13 Oct 2026). Placed in Steering and probing for LLM safety, one of 13 directions in Mechanistic interpretability. Builds on 44 of its 48 references that are in the map. First posted on arXiv 3 Mar 2026.

Builds on

44 of 48 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Programming Refusal with Conditional Activation Steering6 Sep

Other references · 43

  1. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,777
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,550
  3. 2016Gaussian Error Linear Units (GELUs)cited by 3,472
  4. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,407
  5. 2019Root Mean Square Layer Normalizationcited by 1,449
  6. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,286
  7. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,280
  8. 2025Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Modelscited by 1,222
  9. 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,042
  10. 2023Steering Language Models With Activation Engineeringcited by 873
  11. 2023Erasing Concepts from Diffusion Modelscited by 535
  12. 2022Safe Latent Diffusion: Mitigating Inappropriate Degeneration in Diffusion Modelscited by 509
31 more
  1. 2023Unified Concept Editing in Diffusion Modelscited by 384
  2. 2023Forget-Me-Not: Learning to Forget in Text-to-Image Diffusion Modelscited by 301
  3. 2024Infinity: Scaling Bitwise AutoRegressive Modeling for High-Resolution Image Synthesiscited by 300
  4. 2025Z-Image: An Efficient Image Generation Foundation Model with Single-Stream Diffusion Transformercited by 261
  5. 2023Ring-A-Bell! How Reliable are Concept Removal Methods for Diffusion Models?cited by 248
  6. 2024ShieldGemma: Generative AI Content Moderation Based on Gemmacited by 238
  7. 2022Extracting Latent Steering Vectors from Pretrained Language Modelscited by 221
  8. 2023SneakyPrompt: Jailbreaking Text-to-image Generative Modelscited by 187
  9. 2023Unsafe Diffusion: On the Generation of Unsafe Images and Hateful Memes From Text-To-Image Modelscited by 174
  10. 2023Concept Sliders: LoRA Adaptors for Precise Control in Diffusion Modelscited by 151
  11. 2025The Geometry of Refusal in Large Language Models: Concept Cones and Representational Independencecited by 114
  12. 2025SAeUron: Interpretable Concept Unlearning in Diffusion Models with Sparse Autoencoderscited by 79
  13. 2024Latent Guard: a Safety Framework for Text-to-image Generationcited by 70
  14. 2024Unlearning Concepts in Diffusion Model via Concept Domain Correction and Concept Preserving Gradientcited by 69
  15. 2024SafeGen: Mitigating Sexually Explicit Content Generation in Text-to-Image Modelscited by 65
  16. 2024Controlling Language and Diffusion Models by Transporting Activationscited by 65
  17. 2023Concept Algebra for (Score-Based) Text-Controlled Generative Modelscited by 59
  18. 2024LlavaGuard: An Open VLM-based Framework for Safeguarding Vision Datasets and Modelscited by 52
  19. 2025T2ISafety: Benchmark for Assessing Fairness, Toxicity, and Privacy in Image Generationcited by 48
  20. 2024VAR-CLIP: Text-to-Image Generator with Visual Auto-Regressive Modelingcited by 42
  21. 2024Perception-guided Jailbreak against Text-to-Image Modelscited by 40
  22. 2024Adversarial Attacks and Defenses on Text-to-Image Diffusion Models: A Surveycited by 34
  23. 2024Universal Prompt Optimizer for Safe Text-to-Image Generationcited by 27
  24. 2025T2I-RiskyPrompt: A Benchmark for Safety Evaluation, Attack, and Defense on Text-to-Image Modelcited by 17
  25. 2025Reason2Attack: Jailbreaking Text-to-Image Models via LLM Reasoningcited by 15
  26. 2025Side Effects of Erasing Concepts from Diffusion Modelscited by 13
  27. 2025Revoking Amnesia: RL-based Trajectory Optimization to Resurrect Erased Concepts in Diffusion Modelscited by 11
  28. 2025Hessian Geometry of Latent Space in Generative Modelscited by 9
  29. 2025Gandalf the Red: Adaptive Security for LLMscited by 7
  30. 2025AutoPrompt: Automated Red-Teaming of Text-to-Image Models via LLM-Driven Adversarial Promptscited by 6
  31. 2020Provably Robust Metric Learningcited by 3

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026Do Models Share Safety Representations? Cross-Model Steering for Safe Visual Generationcited by 2
  2. 2026SACE: Concept Erasure at the Semantic Singularity in Visual Autoregressive Modelscited by 0
  3. 2026AcFlow: Controlling Text-to-Image Diffusion Transformers via Learned Conditional Activation Flowcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Robust and Generalizable Safety Steering for Text-to-Image Diffusion Transformers
  2. 2026SPOT: Selective Prompt Projection via Total Variation for Inference-Only Safe Text-to-Image Generation
  3. 2026DTVI: Dual-Stage Textual and Visual Intervention for Safe Text-to-Image Generation
  4. 2025Efficient Input-level Backdoor Defense on Text-to-Image Synthesis via Neuron Activation Variation
  5. 2025SafeGuider: Robust and Practical Content Safety Control for Text-to-Image Models
  6. 2026SafeDiffusion-R1: Online Reward Steering for Safe Diffusion Post-Training
  7. 2024Safeguarding Text-to-Image Generation via Inference-Time Prompt-Noise Optimization
  8. 2025Activation Steering Meets Preference Optimization: Defense Against Jailbreaks in Vision Language Models