mlsanityData to 7 Oct 2026

Attention Shift: Steering AI Away from Unsafe Content

arXiv 2410.04447 ↗First posted 6 Oct 2024cs.CV cs.CR cs.LGcited by 0

On the map: LLM safety and security → Vision-language and text-to-image safety

Abstract (arXiv)

This study investigates the generation of unsafe or harmful content in state-of-the-art generative models, focusing on methods for restricting such generations. We introduce a novel training-free approach using attention reweighing to remove unsafe concepts without additional training during inference. We compare our method against existing ablation methods, evaluating the performance on both, direct and adversarial jailbreak prompts, using qualitative and quantitative metrics. We hypothesize potential reasons for the observed results and discuss the limitations and broader implications of content restriction.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Vision-language and text-to-image safety, one of 12 directions in LLM safety and security. Builds on 21 of its 24 references that are in the map. First posted on arXiv 6 Oct 2024.

Builds on

21 of 24 references.

Closest ancestors · 4

Same direction, newest first

  1. 2023To Generate or Not? Safety-Driven Unlearned Diffusion Models Are Still Easy To Generate Unsafe Images ... For Now18 Oct
  2. 2023SneakyPrompt: Jailbreaking Text-to-image Generative Models20 May
  3. 2023Forget-Me-Not: Learning to Forget in Text-to-Image Diffusion Models30 Mar
  4. 2022Red-Teaming the Stable Diffusion Safety Filter3 Oct

Other references · 17

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  4. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,409
  5. 2024Mixtral of Expertscited by 1,785
  6. 2023ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generationcited by 1,010
  7. 2022Safe Latent Diffusion: Mitigating Inappropriate Degeneration in Diffusion Modelscited by 501
  8. 2023Ablating Concepts in Text-to-Image Diffusion Modelscited by 312
  9. 2023Unsafe Diffusion: On the Generation of Unsafe Images and Hateful Memes From Text-To-Image Modelscited by 171
  10. 2023One-Dimensional Adapter to Rule Them All: Concepts, Diffusion Models and Erasing Applicationscited by 164
  11. 2023Receler: Reliable Concept Erasing of Text-to-Image Diffusion Models via Lightweight Eraserscited by 118
  12. 2024Erasing Undesirable Influence in Diffusion Modelscited by 52
5 more
  1. 2023Stable Diffusion Exposed: Gender Bias from Prompt to Imagecited by 25
  2. 2024FAIntbench: A Holistic and Precise Benchmark for Bias Evaluation in Text-to-Image Modelscited by 21
  3. 2024Bias in Generative AIcited by 19
  4. 2024BIGbench: A Unified Benchmark for Evaluating Multi-dimensional Social Biases in Text-to-Image Modelscited by 17
  5. 2024Unmasking the Veil: An Investigation into Concept Ablation for Privacy and Copyright Protection in Imagescited by 2

Built on by

Not cited yet.