mlsanityData to 14 Oct 2026

CapsFake: A Multimodal Capsule Network for Detecting Instruction-Guided Deepfakes

arXiv 2504.19212 ↗First posted 27 Apr 2025cs.CV cs.AIcited by 1

Not on the map. Why?

Abstract (arXiv)

The rapid evolution of deepfake technology, particularly in instruction-guided image editing, threatens the integrity of digital images by enabling subtle, context-aware manipulations. Generated conditionally from real images and textual prompts, these edits are often imperceptible to both humans and existing detection systems, revealing significant limitations in current defenses. We propose a novel multimodal capsule network, CapsFake, designed to detect such deepfake image edits by integrating low-level capsules from visual, textual, and frequency-domain modalities. High-level capsules, predicted through a competitive routing mechanism, dynamically aggregate local features to identify manipulated regions with precision. Evaluated on diverse datasets, including MagicBrush, Unsplash Edits, Open Images Edits, and Multi-turn Edits, CapsFake outperforms state-of-the-art methods by up to 20% in detection accuracy. Ablation studies validate its robustness, achieving detection rates above 94% under natural perturbations and 96% against adversarial attacks, with excellent generalization to unseen editing scenarios. This approach establishes a powerful framework for countering sophisticated image manipulations.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 33 of its 48 references that are in the map. First posted on arXiv 27 Apr 2025.

Builds on

33 of 48 references.

References · 33

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  5. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  6. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  7. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
  8. 2023Segment Anythingcited by 8,234
  9. 2014Explaining and Harnessing Adversarial Examplescited by 7,364
  10. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  11. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
  12. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
21 more
  1. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  2. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,322
  3. 2022A ConvNet for the 2020scited by 3,579
  4. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,876
  5. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,611
  6. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,371
  7. 2019FaceForensics++: Learning to Detect Manipulated Facial Imagescited by 1,101
  8. 2021StyleCLIP: Text-Driven Manipulation of StyleGAN Imagerycited by 991
  9. 2022Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translationcited by 906
  10. 2017Dynamic Routing Between Capsulescited by 689
  11. 2023Towards Universal Fake Image Detectors that Generalize Across Generative Modelscited by 490
  12. 2019On the Detection of Digital Face Manipulationcited by 260
  13. 2023JourneyDB: A Benchmark for Generative Image Understandingcited by 205
  14. 2022DE-FAKE: Detection and Attribution of Fake Images Generated by Text-to-Image Generation Modelscited by 168
  15. 2022Towards the Detection of Diffusion Model Deepfakescited by 111
  16. 2024SEED-Data-Edit Technical Report: A Hybrid Dataset for Instructional Image Editingcited by 97
  17. 2021Deepfakes Generation and Detection: State-of-the-art, open challenges, countermeasures, and way forwardcited by 82
  18. 2022ObjectFormer for Image Manipulation Detection and Localizationcited by 78
  19. 2024Cross-Modality and Within-Modality Regularization for Audio-Visual DeepFake Detectioncited by 24
  20. 2023Detecting Deepfakes Without Seeing Anycited by 17
  21. 2024An Analysis of Recent Advances in Deepfake Image Detection in an Evolving Threat Landscapecited by 16

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.