Explaining AI-Image Detection: What the Heatmap Actually Shows
Not on the map. Why?
Abstract (arXiv)
A marketplace review photograph is a document: platforms approve refunds on it, and generative models drove the cost of forging one to zero. We study that detection problem, so we build a detector and attach an attribution map as its evidence, then measure what that pair delivers on 186,527 images under controls designed to change our conclusions when something is wrong. Compression history, not synthesis, drives naive evaluation: our strongest model reaches 0.9999 PR-AUC (area under the precision-recall curve) on a product-disjoint split, yet falls to 0.7254 once we re-encode synthetics into the real class's format, while five public detectors move by at most 0.07. Aligning one class relocates the cue rather than removing it, and the repaired model then assigns native files a median probability of synthesis of 0.0004. One identical final encode for both classes repairs that, and a three-seed factorial credits the encoding change with the whole gain (+0.176 +- 0.009 PR-AUC). That encode equalises the last stage only: forensic features alone still separate the classes at 0.7145 against a base rate of 0.254. For evidence we test maps causally, against controls that never consult the detector. Whether an attribution ranking exists at all depends on whether the detector reacts to the image. On our first-fix detector, which calls 96 of 100 edited frames real, no map beats a random one. On the detector we selected, twelve of seventeen maps clear that control on edited images and eight on generated ones; perturbation leads both axes and no gradient-CAM variant shows a positive advantage. The trivial controls never clear it, and on generated images the centre prior is worse than random. Our ensembled regional map clears both axes and takes the top pixel AP at 12.4 s per map against 44.9 for occlusion. Clearing a detector-blind control is not yet a faithful explanation, and we demonstrate none.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 56 of its 66 references that are in the map. First posted on arXiv 31 Jul 2026.
Builds on
56 of 66 references.
References · 56
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
- 2023Segment Anythingcited by 8,158
- 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
- 2017A Unified Approach to Interpreting Model Predictionscited by 6,488
- 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,040
- 2017On Calibration of Modern Neural Networkscited by 4,860
- 2017Axiomatic Attribution for Deep Networkscited by 3,350
- 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,048
- 2013Visualizing and Understanding Convolutional Networkscited by 2,278
- 2018Sanity Checks for Saliency Mapscited by 1,149
- 2017SmoothGrad: removing noise by adding noisecited by 1,097
38 more
- 2017Grad-CAM++: Improved Visual Explanations for Deep Convolutional Networkscited by 936
- 2019CNN-generated images are surprisingly easy to spot... for nowcited by 842
- 2018RISE: Randomized Input Sampling for Explanation of Black-box Modelscited by 825
- 2020Quantifying Attention Flow in Transformerscited by 759
- 2020Transformer Interpretability Beyond Attention Visualizationcited by 620
- 2023MagicBrush: A Manually Annotated Dataset for Instruction-Guided Image Editingcited by 615
- 2019Score-CAM: Score-Weighted Visual Explanations for Convolutional Neural Networkscited by 575
- 2019ERASER: A Benchmark to Evaluate Rationalized NLP Modelscited by 523
- 2023Towards Universal Fake Image Detectors that Generalize Across Generative Modelscited by 483
- 2018A Benchmark for Interpretability Methods in Deep Neural Networkscited by 467
- 2017Interpretation of Neural Networks is Fragilecited by 443
- 2017The (Un)reliability of saliency methodscited by 331
- 2016Top-down Neural Attention by Excitation Backpropcited by 318
- 2023GenImage: A Million-Scale Benchmark for Detecting AI-Generated Imagecited by 318
- 2025Perception Encoder: The best visual embeddings are not at the output of the networkcited by 315
- 2019Counterfactual Visual Explanationscited by 312
- 2019On the (In)fidelity and Sensitivity for Explanationscited by 309
- 2022On the detection of synthetic images generated by diffusion modelscited by 282
- 2019Understanding Deep Networks via Extremal Perturbations and Smooth Maskscited by 257
- 2016What do different evaluation metrics tell us about saliency models?cited by 184
- 2022Quantus: An Explainable AI Toolkit for Responsible Evaluation of Neural Network Explanations and Beyondcited by 171
- 2022DE-FAKE: Detection and Attribution of Fake Images Generated by Text-to-Image Generation Modelscited by 167
- 2023Raising the Bar of AI-generated Image Detection with CLIPcited by 163
- 2022TruFor: Leveraging all-round clues for trustworthy image forgery detection and localizationcited by 130
- 2019Sanity Checks for Saliency Metricscited by 119
- 2024SIDA: Social Media Image Deepfake Detection, Localization and Explanation with Large Multimodal Modelcited by 107
- 2020Evaluating Weakly Supervised Object Localization Methods Rightcited by 104
- 2022A Consistent and Efficient Evaluation Strategy for Attribution Methodscited by 100
- 2021Learning JPEG Compression Artifacts for Image Manipulation Detection and Localizationcited by 78
- 2025Dual Data Alignment Makes AI-Generated Image Detector Easier Generalizablecited by 58
- 2023AutoSplice: A Text-prompt Manipulated Image Dataset for Media Forensicscited by 46
- 2024Fake or JPEG? Revealing Common Biases in Generated Image Detection Datasetscited by 43
- 2023IML-ViT: Benchmarking Image Manipulation Localization by Vision Transformercited by 42
- 2023The Meta-Evaluation Problem in Explainable AI: Identifying Reliable Estimators with MetaQuantuscited by 37
- 2025AIGI-Holmes: Towards Explainable and Generalizable AI-Generated Image Detection via Multimodal Large Language Modelscited by 33
- 2024LeGrad: An Explainability Method for Vision Transformers via Feature Formation Sensitivitycited by 32
- 2022ViT-CX: Causal Explanation of Vision Transformerscited by 27
- 2023Weakly-supervised deepfake localization in diffusion-generated imagescited by 24
50 most cited of 56 All 56 (JSON)
Built on by
Not cited yet.