mlsanityData to 7 Oct 2026

Leveraging Representations from Intermediate Encoder-blocks for Synthetic Image Detection

arXiv 2402.19091 ↗First posted 29 Feb 2024cs.CVcited by 88

Not on the map. Why?

Abstract (arXiv)

The recently developed and publicly available synthetic image generation methods and services make it possible to create extremely realistic imagery on demand, raising great risks for the integrity and safety of online information. State-of-the-art Synthetic Image Detection (SID) research has led to strong evidence on the advantages of feature extraction from foundation models. However, such extracted features mostly encapsulate high-level visual semantics instead of fine-grained details, which are more important for the SID task. On the contrary, shallow layers encode low-level visual information. In this work, we leverage the image representations extracted by intermediate Transformer blocks of CLIP's image-encoder via a lightweight network that maps them to a learnable forgery-aware vector space capable of generalizing exceptionally well. We also employ a trainable module to incorporate the importance of each Transformer block to the final prediction. Our method is compared against the state-of-the-art by evaluating it on 20 test datasets and exhibits an average +10.6% absolute performance improvement. Notably, the best performing models require just a single epoch for training (~8 minutes). Code available at https://github.com/mever-team/rine.

Key facts. Cited by 88 papers in the map (data through 7 Oct 2026). Builds on 44 of its 60 references that are in the map. First posted on arXiv 29 Feb 2024.

Builds on

44 of 60 references.

References · 44

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  5. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  6. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  7. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  8. 2016Layer Normalizationcited by 5,579
  9. 2016Feature Pyramid Networks for Object Detectioncited by 5,021
  10. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  11. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  12. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
32 more
  1. 2016Gaussian Error Linear Units (GELUs)cited by 3,431
  2. 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,967
  3. 2020Supervised Contrastive Learningcited by 2,803
  4. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  5. 2016Xception: Deep Learning with Depthwise Separable Convolutionscited by 2,320
  6. 2019Semantic Image Synthesis with Spatially-Adaptive Normalizationcited by 1,295
  7. 2019FaceForensics++: Learning to Detect Manipulated Facial Imagescited by 1,095
  8. 2017StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translationcited by 1,036
  9. 2022Diffusion Models in Vision: A Surveycited by 1,008
  10. 2019CNN-generated images are surprisingly easy to spot... for nowcited by 842
  11. 2020Thinking in Frequency: Face Forgery Detection by Mining Frequency-aware Cluescited by 518
  12. 2020Leveraging Frequency Analysis for Deep Fake Image Recognitioncited by 501
  13. 2023Towards Universal Fake Image Detectors that Generalize Across Generative Modelscited by 483
  14. 2018Learning to See in the Darkcited by 459
  15. 2017Deep Layer Aggregationcited by 436
  16. 2019Detecting and Simulating Artifacts in GAN Fake Imagescited by 293
  17. 2022On the detection of synthetic images generated by diffusion modelscited by 282
  18. 2020What makes fake images detectable? Understanding properties that generalizecited by 272
  19. 2020The Creation and Detection of Deepfakes: A Surveycited by 246
  20. 2020DeepFakes and Beyond: A Survey of Face Manipulation and Fake Detectioncited by 231
  21. 2017Photographic Image Synthesis with Cascaded Refinement Networkscited by 223
  22. 2018Do GANs leave artificial fingerprints?cited by 188
  23. 2019Detecting GAN generated Fake Images using Co-occurrence Matricescited by 124
  24. 2023Intriguing properties of synthetic images: from generative adversarial networks to diffusion modelscited by 121
  25. 2018ForensicTransfer: Weakly-supervised Domain Adaptation for Forgery Detectioncited by 104
  26. 2022FrePGAN: Robust Deepfake Detection Using Frequency-level Perturbationscited by 75
  27. 2023Seeing is not always believing: Benchmarking Human and Model Perception of AI-Generated Imagescited by 65
  28. 2018Diverse Image Synthesis from Semantic Layouts via Conditional IMLEcited by 60
  29. 2021BiHPF: Bilateral High-Pass Filters for Robust Deepfake Detectioncited by 60
  30. 2022A survey on GANs for computer vision: Recent research, analysis and taxonomycited by 15
  31. 2023Improving Synthetically Generated Image Detection in Cross-Concept Settingscited by 15
  32. 2022Aggregating Layers for Deepfake Detectioncited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 14; 2025: 27; 2026: 47.

88 papers.

Sort:

By area:

  1. 2024Improving Synthetic Image Detection Towards Generalization: An Image Transformation Perspectivecited by 88
  2. 2024A Bias-Free Training Paradigm for More General AI-generated Image Detectioncited by 67
  3. 2024Any-Resolution AI-Generated Image Detection by Spectral Learningcited by 56
  4. 2024FakeBench: Probing Explainable Fake Image Detection via Large Multimodal Modelscited by 36
  5. 2025AIGI-Holmes: Towards Explainable and Generalizable AI-Generated Image Detection via Multimodal Large Language Modelscited by 33
  6. 2025Bridging the Gap Between Ideal and Real-world Evaluation: Benchmarking AI-Generated Image Detection in Challenging Scenarioscited by 24
  7. 2025Seeing What Matters: Generalizable AI-generated Video Detection with Forensic-Oriented Augmentationcited by 23
  8. 2024TGIF: Text-Guided Inpainting Forgery Datasetcited by 20
  9. 2024DeCLIP: Decoding CLIP representations for deepfake localizationcited by 17
  10. 2025Breaking Latent Prior Bias in Detectors for Generalizable AIGC Image Detectioncited by 16
  11. 2024SIDBench: A Python Framework for Reliably Assessing Synthetic Image Detection Methodscited by 14
  12. 2024ForgeLens: Data-Efficient Forgery Focus for Generalizable Forgery Image Detectioncited by 14
38 more
  1. 2025Methods and Trends in Detecting AI-Generated Images: A Comprehensive Reviewcited by 12
  2. 2025AI-GenBench: A New Ongoing Benchmark for AI-Generated Image Detectioncited by 10
  3. 2024TextureCrop: Enhancing Synthetic Image Detection through Texture-based Croppingcited by 9
  4. 2024Evolution of Detection Performance throughout the Online Lifespan of Synthetic Imagescited by 8
  5. 2025Navigating the Challenges of AI-Generated Image Detection in the Wild: What Truly Matters?cited by 8
  6. 2025Towards Generalizable AI-Generated Image Detection via Image-Adaptive Prompt Learningcited by 8
  7. 2026Preserving Forgery Artifacts: AI-Generated Video Detection at Native Scalecited by 8
  8. 2026Simplicity Prevails: The Emergence of Generalizable AIGI Detection in Visual Foundation Modelscited by 6
  9. 2025AI-Generated Image Detection: An Empirical Study and Future Research Directionscited by 5
  10. 2025Fully AI-Generated Image Detection: Definition, Recent Advances and Challengescited by 4
  11. 2025RAID: A Dataset for Testing the Adversarial Robustness of AI-Generated Image Detectorscited by 4
  12. 2025DGS-Net: Distillation-Guided Gradient Surgery for CLIP Fine-Tuning in AI-Generated Image Detectioncited by 4
  13. 2026Layer Consistency Matters: Elegant Latent Transition Discrepancy for Generalizable Synthetic Image Detectioncited by 4
  14. 2026Where Detectors Fail: Probing Generative Space for Generalizable AI-Generated Image Detectioncited by 4
  15. 2024Present and Future Generalization of Synthetic Image Detectorscited by 3
  16. 2026Unveiling Perceptual Artifacts: A Fine-Grained Benchmark for Interpretable AI-Generated Image Detectioncited by 3
  17. 2026AgentFoX: LLM Agent-Guided Fusion with eXplainability for AI-Generated Image Detectioncited by 3
  18. 2026Diversity Matters: Dataset Diversification and Dual-Branch Network for Generalized AI-Generated Image Detectioncited by 3
  19. 2026FakeVLM-R1: Internalizing Physical Laws via CoT for Synthetic Image Detectioncited by 3
  20. 2024Learning on Less: Constraining Pre-trained Model Learning for Generalizable Diffusion-Generated Image Detectioncited by 2
  21. 2025Leveraging Failed Samples: A Few-Shot and Training-Free Framework for Generalized Deepfake Detectioncited by 2
  22. 2025"Humor, Art, or Misinformation?": A Multimodal Dataset for Intent-Aware Synthetic Image Detectioncited by 2
  23. 2025Generalized Design Choices for Deepfake Detectorscited by 2
  24. 2026Do Transformations Reveal the Truth? Generative Residual Learning for Generalized AI-Generated Image Detectioncited by 2
  25. 2024The Visual Counter Turing Test (VCT2): A Benchmark for Evaluating AI-Generated Image Detection and the Visual AI Index (VAI)cited by 1
  26. 2025Few-Shot Class-Incremental Model Attribution Using Learnable Representation From CLIP-ViT Featurescited by 1
  27. 2025Autonomous and Self-Adapting System for Synthetic Media Detection and Attributioncited by 1
  28. 2025Unraveling Hidden Representations: A Multi-Modal Layer Analysis for Better Synthetic Content Forensicscited by 1
  29. 2025GAMMA: Generalizable Alignment via Multi-task and Manipulation-Augmented Training for AI-Generated Image Detectioncited by 1
  30. 2025PolyJuice Makes It Real: Black-Box, Universal Red Teaming for Synthetic Image Detectorscited by 1
  31. 2025Investigating self-supervised representations for audio-visual deepfake detectioncited by 1
  32. 2025PRADA: Probability-Ratio-Based Attribution and Detection of Autoregressive-Generated Imagescited by 1
  33. 2026Detecting AI-Generated Images via Distributional Deviations from Real Imagescited by 1
  34. 2026Your AI-Generated Image Detector Can Secretly Achieve SOTA Accuracy, If Calibratedcited by 1
  35. 2026Synthetic Image Detection with CLIP: Understanding and Assessing Predictive Cuescited by 1
  36. 2026The Synthetic Media Shift: Tracking the Rise, Virality, and Detectability of AI-Generated Multimodal Misinformationcited by 1
  37. 2026TAP into the Patch Tokens: Leveraging Vision Foundation Model Features for AI-Generated Image Detectioncited by 1
  38. 2026Frequency-Aware Semantic Fusion with Gated Injection for AI-generated Image Detectioncited by 1

50 most cited of 88 All 88 (JSON)