mlsanityData to 14 Oct 2026

Perceptual Artifacts Localization for Image Synthesis Tasks

arXiv 2310.05590 ↗First posted 9 Oct 2023cs.CVcited by 43

Not on the map. Why?

Abstract (arXiv)

Recent advancements in deep generative models have facilitated the creation of photo-realistic images across various tasks. However, these generated images often exhibit perceptual artifacts in specific regions, necessitating manual correction. In this study, we present a comprehensive empirical examination of Perceptual Artifacts Localization (PAL) spanning diverse image synthesis endeavors. We introduce a novel dataset comprising 10,168 generated images, each annotated with per-pixel perceptual artifact labels across ten synthesis tasks. A segmentation model, trained on our proposed dataset, effectively localizes artifacts across a range of tasks. Additionally, we illustrate its proficiency in adapting to previously unseen models using minimal training samples. We further propose an innovative zoom-in inpainting pipeline that seamlessly rectifies perceptual artifacts in the generated images. Through our experimental analyses, we elucidate several practical downstream applications, such as automated artifact rectification, non-referential image quality evaluation, and abnormal region detection in images. The dataset and code are released.

Key facts. Cited by 43 papers in the map (data through 14 Oct 2026). Builds on 59 of its 77 references that are in the map. First posted on arXiv 9 Oct 2023.

Builds on

59 of 77 references.

References · 59

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  4. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  5. 2020Denoising Diffusion Implicit Modelscited by 9,781
  6. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  7. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,733
  8. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  9. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  10. 2022Scalable Diffusion Models with Transformerscited by 6,054
  11. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
  12. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,859
38 more
  1. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,614
  2. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,588
  3. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,626
  4. 2018Glow: Generative Flow with Invertible 1x1 Convolutionscited by 1,545
  5. 2022RePaint: Inpainting using Denoising Diffusion Probabilistic Modelscited by 1,430
  6. 2019Deep High-Resolution Representation Learning for Visual Recognitioncited by 1,420
  7. 2015LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loopcited by 1,355
  8. 2021Palette: Image-to-Image Diffusion Modelscited by 1,300
  9. 2019Semantic Image Synthesis with Spatially-Adaptive Normalizationcited by 1,299
  10. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,295
  11. 2021Alias-Free Generative Adversarial Networkscited by 1,184
  12. 2019FaceForensics++: Learning to Detect Manipulated Facial Imagescited by 1,101
  13. 2021MUSIQ: Multi-scale Image Quality Transformercited by 1,081
  14. 2021Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Datacited by 970
  15. 2022eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoiserscited by 931
  16. 2016COCO-Stuff: Thing and Stuff Classes in Contextcited by 864
  17. 2019CNN-generated images are surprisingly easy to spot... for nowcited by 847
  18. 2021Resolution-robust Large Mask Inpainting with Fourier Convolutionscited by 738
  19. 2020Encoding in Style: a StyleGAN Encoder for Image-to-Image Translationcited by 727
  20. 2020Leveraging Frequency Analysis for Deep Fake Image Recognitioncited by 504
  21. 2023Towards Universal Fake Image Detectors that Generalize Across Generative Modelscited by 490
  22. 2018MesoNet: a Compact Facial Video Forgery Detection Networkcited by 409
  23. 2021Multi-attentional Deepfake Detectioncited by 341
  24. 2022CogView2: Faster and Better Text-to-Image Generation via Hierarchical Transformerscited by 307
  25. 2019Detecting and Simulating Artifacts in GAN Fake Imagescited by 293
  26. 2020What makes fake images detectable? Understanding properties that generalizecited by 273
  27. 2021Spatial-Phase Shallow Learning: Rethinking Face Forgery Detection in Frequency Domaincited by 244
  28. 2021Large Scale Image Completion via Co-Modulated Generative Adversarial Networkscited by 220
  29. 2022MAT: Mask-Aware Transformer for Large Hole Image Inpaintingcited by 218
  30. 2022The Role of ImageNet Classes in Fr'echet Inception Distancecited by 213
  31. 2022Dual Diffusion Implicit Bridges for Image-to-Image Translationcited by 213
  32. 2022Versatile Diffusion: Text, Images and Variations All in One Diffusion Modelcited by 199
  33. 2022Pretraining is All You Need for Image-to-Image Translationcited by 160
  34. 2019Detecting GAN generated Fake Images using Co-occurrence Matricescited by 124
  35. 2018ForensicTransfer: Weakly-supervised Domain Adaptation for Forgery Detectioncited by 104
  36. 2022Any-resolution Training for High-resolution Image Synthesiscited by 72
  37. 2017Recasting Residual-based Local Descriptors as Convolutional Neural Networks: an Application to Image Forgery Detectioncited by 56
  38. 2019Detecting Photoshopped Faces by Scripting Photoshopcited by 55

50 most cited of 59 All 59 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 3; 2024: 11; 2025: 18; 2026: 11.

43 papers.

Sort:

By area:

  1. 2023Rich Human Feedback for Text-to-Image Generationcited by 141
  2. 2024SIDA: Social Media Image Deepfake Detection, Localization and Explanation with Large Multimodal Modelcited by 108
  3. 2024Detecting Multimedia Generated by Large AI Models: A Surveycited by 76
  4. 2025Spot the Fake: Large Multimodal Model-Based Synthetic Image Detection with Artifact Explanationcited by 68
  5. 2025LEGION: Learning to Ground and Explain for Synthetic Image Detectioncited by 65
  6. 2023Amodal Completion via Progressive Mixed Context Diffusioncited by 38
  7. 2024FakeBench: Probing Explainable Fake Image Detection via Large Multimodal Modelscited by 36
  8. 2025Ivy-Fake: A Unified Explainable Framework and Benchmark for Image and Video AIGC Detectioncited by 34
  9. 2025DenseDPO: Fine-Grained Temporal Preference Optimization for Video Diffusion Modelscited by 34
  10. 2024A Survey on Quality Metrics for Text-to-Image Generationcited by 18
  11. 2025MMIG-Bench: Towards Comprehensive and Explainable Evaluation of Multi-Modal Image Generation Modelscited by 17
  12. 2024A Survey of Defenses Against AI-Generated Visual Media: Detection,Disruption, and Authenticationcited by 13
31 more
  1. 2024Detecting Human Artifacts from Text-to-Image Modelscited by 12
  2. 2025VLForgery Face Triad: Detection, Localization and Attribution via Multimodal Large Language Modelscited by 12
  3. 2025RealGen: Photorealistic Text-to-Image Generation via Detector-Guided Rewardscited by 12
  4. 2025Silent Branding Attack: Trigger-free Data Poisoning Attack on Text-to-Image Diffusion Modelscited by 11
  5. 2025Focus-N-Fix: Region-Aware Fine-Tuning for Text-to-Image Generationcited by 9
  6. 2025Temporal Score Analysis for Understanding and Correcting Diffusion Artifactscited by 9
  7. 2023Tracing Hyperparameter Dependencies for Model Parsing via Learnable Graph Pooling Networkcited by 8
  8. 2025DiffDoctor: Diagnosing Image Diffusion Models Before Treatingcited by 7
  9. 2024Self-Adaptive Reality-Guided Diffusion for Artifact-Free Super-Resolutioncited by 6
  10. 2024Refine-by-Align: Reference-Guided Artifacts Refinement through Semantic Alignmentcited by 5
  11. 2025So-Fake: Benchmarking and Explaining Social Media Image Forgery Detectioncited by 5
  12. 2024Puzzle Similarity: A Perceptually-guided Cross-Reference Metric for Artifact Detection in 3D Scene Reconstructionscited by 3
  13. 2026Unveiling Perceptual Artifacts: A Fine-Grained Benchmark for Interpretable AI-Generated Image Detectioncited by 3
  14. 2024Similarity Trajectories: Linking Sampling Process to Artifacts in Diffusion-Generated Imagescited by 2
  15. 2025Efficient High-Resolution Image Editing with Hallucination-Aware Loss and Adaptive Tilingcited by 2
  16. 2025BLANKET: Anonymizing Faces in Infant Video Recordingscited by 2
  17. 2026SR-Prominence: A Crowdsourced Protocol and Dataset Suite for Perceptually-Weighted Super-Resolution Artifact Evaluationcited by 2
  18. 2024Uncertain Boundaries: Multidisciplinary Approaches to Copyright Issues in Generative AIcited by 1
  19. 2025Prominence-Aware Artifact Detection and Dataset for Image Super-Resolutioncited by 1
  20. 2025NeXT-IMDL: Build Benchmark for NeXT-Generation Image Manipulation Detection & Localizationcited by 1
  21. 2026DIAMOND: Directed Inference for Artifact Mitigation in Flow Matching Modelscited by 1
  22. 2026See and Fix the Flaws: Enabling VLMs and Diffusion Models to Comprehend Visual Artifacts via Agentic Data Synthesiscited by 1
  23. 2026SR-Ground: Image Quality Grounding for Super-Resolved Contentcited by 1
  24. 2026SalArt-VQA: Diagnosing Whether VLMs Understand Salient Artifacts in Generated Imagescited by 1
  25. 2026Grounding and Explaining Visual Evidence for AI-Generated Image Detection in Human-Centric Scenescited by 1
  26. 2025Prefill-Guided Thinking for zero-shot detection of AI-generated imagescited by 0
  27. 2025Image Realness Assessment and Localization with Multimodal Featurescited by 0
  28. 2026PR-IQA: Partial-Reference Image Quality Assessment for Diffusion-Based Novel View Synthesiscited by 0
  29. 2026Skill-Aligned Annotation for Reliable Evaluation in Text-to-Image Generationcited by 0
  30. 2026Action Forcing: Training World Models on Unsupervised Video by Recovering Underlying Egomotion Basescited by 0
  31. 2026VIEScore2: Unified Image Evaluation with Spatially Grounded Explanationscited by 0