mlsanityData to 7 Oct 2026

Explaining AI-Image Detection: What the Heatmap Actually Shows

arXiv 2607.29581 ↗First posted 31 Jul 2026cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

A marketplace review photograph is a document: platforms approve refunds on it, and generative models drove the cost of forging one to zero. We study that detection problem, so we build a detector and attach an attribution map as its evidence, then measure what that pair delivers on 186,527 images under controls designed to change our conclusions when something is wrong. Compression history, not synthesis, drives naive evaluation: our strongest model reaches 0.9999 PR-AUC (area under the precision-recall curve) on a product-disjoint split, yet falls to 0.7254 once we re-encode synthetics into the real class's format, while five public detectors move by at most 0.07. Aligning one class relocates the cue rather than removing it, and the repaired model then assigns native files a median probability of synthesis of 0.0004. One identical final encode for both classes repairs that, and a three-seed factorial credits the encoding change with the whole gain (+0.176 +- 0.009 PR-AUC). That encode equalises the last stage only: forensic features alone still separate the classes at 0.7145 against a base rate of 0.254. For evidence we test maps causally, against controls that never consult the detector. Whether an attribution ranking exists at all depends on whether the detector reacts to the image. On our first-fix detector, which calls 96 of 100 edited frames real, no map beats a random one. On the detector we selected, twelve of seventeen maps clear that control on edited images and eight on generated ones; perturbation leads both axes and no gradient-CAM variant shows a positive advantage. The trivial controls never clear it, and on generated images the centre prior is worse than random. Our ensembled regional map clears both axes and takes the top pixel AP at 12.4 s per map against 44.9 for occlusion. Clearing a detector-blind control is not yet a faithful explanation, and we demonstrate none.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 56 of its 66 references that are in the map. First posted on arXiv 31 Jul 2026.

Builds on

56 of 66 references.

References · 56

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2023Segment Anythingcited by 8,158
  4. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
  5. 2017A Unified Approach to Interpreting Model Predictionscited by 6,488
  6. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,040
  7. 2017On Calibration of Modern Neural Networkscited by 4,860
  8. 2017Axiomatic Attribution for Deep Networkscited by 3,350
  9. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,048
  10. 2013Visualizing and Understanding Convolutional Networkscited by 2,278
  11. 2018Sanity Checks for Saliency Mapscited by 1,149
  12. 2017SmoothGrad: removing noise by adding noisecited by 1,097
38 more
  1. 2017Grad-CAM++: Improved Visual Explanations for Deep Convolutional Networkscited by 936
  2. 2019CNN-generated images are surprisingly easy to spot... for nowcited by 842
  3. 2018RISE: Randomized Input Sampling for Explanation of Black-box Modelscited by 825
  4. 2020Quantifying Attention Flow in Transformerscited by 759
  5. 2020Transformer Interpretability Beyond Attention Visualizationcited by 620
  6. 2023MagicBrush: A Manually Annotated Dataset for Instruction-Guided Image Editingcited by 615
  7. 2019Score-CAM: Score-Weighted Visual Explanations for Convolutional Neural Networkscited by 575
  8. 2019ERASER: A Benchmark to Evaluate Rationalized NLP Modelscited by 523
  9. 2023Towards Universal Fake Image Detectors that Generalize Across Generative Modelscited by 483
  10. 2018A Benchmark for Interpretability Methods in Deep Neural Networkscited by 467
  11. 2017Interpretation of Neural Networks is Fragilecited by 443
  12. 2017The (Un)reliability of saliency methodscited by 331
  13. 2016Top-down Neural Attention by Excitation Backpropcited by 318
  14. 2023GenImage: A Million-Scale Benchmark for Detecting AI-Generated Imagecited by 318
  15. 2025Perception Encoder: The best visual embeddings are not at the output of the networkcited by 315
  16. 2019Counterfactual Visual Explanationscited by 312
  17. 2019On the (In)fidelity and Sensitivity for Explanationscited by 309
  18. 2022On the detection of synthetic images generated by diffusion modelscited by 282
  19. 2019Understanding Deep Networks via Extremal Perturbations and Smooth Maskscited by 257
  20. 2016What do different evaluation metrics tell us about saliency models?cited by 184
  21. 2022Quantus: An Explainable AI Toolkit for Responsible Evaluation of Neural Network Explanations and Beyondcited by 171
  22. 2022DE-FAKE: Detection and Attribution of Fake Images Generated by Text-to-Image Generation Modelscited by 167
  23. 2023Raising the Bar of AI-generated Image Detection with CLIPcited by 163
  24. 2022TruFor: Leveraging all-round clues for trustworthy image forgery detection and localizationcited by 130
  25. 2019Sanity Checks for Saliency Metricscited by 119
  26. 2024SIDA: Social Media Image Deepfake Detection, Localization and Explanation with Large Multimodal Modelcited by 107
  27. 2020Evaluating Weakly Supervised Object Localization Methods Rightcited by 104
  28. 2022A Consistent and Efficient Evaluation Strategy for Attribution Methodscited by 100
  29. 2021Learning JPEG Compression Artifacts for Image Manipulation Detection and Localizationcited by 78
  30. 2025Dual Data Alignment Makes AI-Generated Image Detector Easier Generalizablecited by 58
  31. 2023AutoSplice: A Text-prompt Manipulated Image Dataset for Media Forensicscited by 46
  32. 2024Fake or JPEG? Revealing Common Biases in Generated Image Detection Datasetscited by 43
  33. 2023IML-ViT: Benchmarking Image Manipulation Localization by Vision Transformercited by 42
  34. 2023The Meta-Evaluation Problem in Explainable AI: Identifying Reliable Estimators with MetaQuantuscited by 37
  35. 2025AIGI-Holmes: Towards Explainable and Generalizable AI-Generated Image Detection via Multimodal Large Language Modelscited by 33
  36. 2024LeGrad: An Explainability Method for Vision Transformers via Feature Formation Sensitivitycited by 32
  37. 2022ViT-CX: Causal Explanation of Vision Transformerscited by 27
  38. 2023Weakly-supervised deepfake localization in diffusion-generated imagescited by 24

50 most cited of 56 All 56 (JSON)

Built on by

Not cited yet.