mlsanityData to 7 Oct 2026

Position: AI/ML Deepfake Research is Misaligned with AI-Generated Non-Consensual Intimate Imagery (AIG-NCII)

arXiv 2607.18263 ↗First posted 16 May 2026cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

AI-generated non-consensual intimate imagery (AIG-NCII) is not adequately addressed in AI/ML literature regarding AI-generated media, commonly referred to as "deepfakes". While research on deepfakes currently focuses on its epistemic harms -- or harms relating to truth and authenticity -- this is misaligned with the dominant reality of generative AI abuse involving sexualized imagery. We conduct a landscape analysis of highly-cited works to demonstrate that technical interventions addressing deepfakes almost entirely ignore AIG-NCII, limiting the research ecosystem to authenticity detection tools. In this position paper, we argue that existing interventions address viewer-centric epistemic harms, such as fraud or scams, but ignore subject-centric dignity harms, such as AIG-NCII. We illustrate that knowing an image is synthetic does not mitigate harms to subjects and may, in some cases, even exacerbate them. We conclude by offering recommendations to realign the field, including updating threat models to consider subject-centric harms and addressing AIG-NCII in AI safety research. Finally, we caution that researchers should only engage in this high-risk domain if they implement safety guardrails for both subjects and researchers and establish partnerships with domain experts in sexual violence prevention.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 58 of its 123 references that are in the map. First posted on arXiv 16 May 2026.

Builds on

58 of 123 references.

References · 58

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  4. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  5. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,510
  6. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  7. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,286
  8. 2018Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examplescited by 1,259
  9. 2017High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANscited by 1,176
  10. 2021Ethical and social risks of harm from Language Modelscited by 957
  11. 2023Erasing Concepts from Diffusion Modelscited by 530
  12. 2022Safe Latent Diffusion: Mitigating Inappropriate Degeneration in Diffusion Modelscited by 501
38 more
  1. 2023Towards Universal Fake Image Detectors that Generalize Across Generative Modelscited by 483
  2. 2023DIRE for Diffusion-Generated Image Detectioncited by 386
  3. 2023The Stable Signature: Rooting Watermarks in Latent Diffusion Modelscited by 289
  4. 2022On the detection of synthetic images generated by diffusion modelscited by 282
  5. 2023Forgery-aware Adaptive Transformer for Generalizable Synthetic Image Detectioncited by 180
  6. 2023Tree-Ring Watermarks: Fingerprints for Diffusion Images that are Invisible and Robustcited by 179
  7. 2023A Recipe for Watermarking Diffusion Modelscited by 136
  8. 2024Gaussian Shading: Provable Performance-Lossless Image Watermarking for Diffusion Modelscited by 131
  9. 2020DeepFaceLab: Integrated, flexible and extensible face-swapping frameworkcited by 129
  10. 2024AEROBLADE: Training-Free Detection of Latent Diffusion Images Using Autoencoder Reconstruction Errorcited by 123
  11. 2022Towards the Detection of Diffusion Model Deepfakescited by 111
  12. 2021Do Datasets Have Politics? Disciplinary Values in Computer Vision Dataset Developmentcited by 85
  13. 2025Spot the Fake: Large Multimodal Model-Based Synthetic Image Detection with Artifact Explanationcited by 66
  14. 2025LEGION: Learning to Ground and Explain for Synthetic Image Detectioncited by 65
  15. 2023The Gradient of Generative AI Release: Methods and Considerationscited by 63
  16. 2018ReDMark: Framework for Residual Diffusion Watermarking on Deep Networkscited by 55
  17. 2023Generalizable Synthetic Image Detection via Language-guided Contrastive Learningcited by 48
  18. 2023Exposing the Fake: Effective Diffusion-Generated Images Detectioncited by 48
  19. 2023MorDIFF: Recognition Vulnerability and Attack Detectability of Face Morphing Attacks Created by Diffusion Autoencoderscited by 44
  20. 2024On Learning Multi-Modal Forgery Representation for Diffusion Generated Video Detectioncited by 40
  21. 2024Adversarial Perturbations Cannot Reliably Protect Artists From Generative AIcited by 38
  22. 2023Robustness and Generalizability of Deepfake Detection: A Study with Diffusion Modelscited by 36
  23. 2023ArtiFact: A Large-Scale Dataset with Artificial and Factual Images for Generalizable and Robust Synthetic Image Detectioncited by 35
  24. 2023Watermarking Diffusion Modelcited by 34
  25. 2024Attack-Resilient Image Watermarking Using Stable Diffusioncited by 33
  26. 2023Open-Sourcing Highly Capable Foundation Models: An evaluation of risks, benefits, and alternative methods for pursuing open-source objectivescited by 31
  27. 2023Deep Image Fingerprint: Towards Low Budget Synthetic Image Detection and Model Lineage Analysiscited by 29
  28. 2024DiffusionFake: Enhancing Generalization in Deepfake Detection via Guided Stable Diffusioncited by 29
  29. 2023Diffusion Noise Feature: Accurate and Fast Generated Image Detectioncited by 21
  30. 2024DiffuseTrace: A Transparent and Flexible Watermarking Scheme for Latent Diffusion Modelcited by 21
  31. 2024Bi-LORA: A Vision-Language Approach for Synthetic Image Detectioncited by 20
  32. 2025AI Safety for Everyonecited by 14
  33. 2023Face Morphing Attack Detection with Denoising Diffusion Probabilistic Modelscited by 12
  34. 2025AdvPaint: Protecting Images from Inpainting Manipulation via Adversarial Attention Disruptioncited by 11
  35. 2025AI Safety Should Prioritize the Future of Workcited by 11
  36. 2024DiffVax: Optimization-Free Image Immunization Against Diffusion-Based Editingcited by 10
  37. 2025AI Automatons: AI Systems Intended to Imitate Humanscited by 10
  38. 2024Analyzing the AI Nudification Application Ecosystemcited by 9

50 most cited of 58 All 58 (JSON)

Built on by

Not cited yet.