mlsanityData to 14 Oct 2026

"Humor, Art, or Misinformation?": A Multimodal Dataset for Intent-Aware Synthetic Image Detection

arXiv 2508.20670 ↗First posted 28 Aug 2025cs.CV cs.MMcited by 2

Not on the map. Why?

Abstract (arXiv)

Recent advances in multimodal AI have enabled progress in detecting synthetic and out-of-context content. However, existing efforts largely overlook the intent behind AI-generated images. To fill this gap, we introduce S-HArM, a multimodal dataset for intent-aware classification, comprising 9,576 "in the wild" image-text pairs from Twitter/X and Reddit, labeled as Humor/Satire, Art, or Misinformation. Additionally, we explore three prompting strategies (image-guided, description-guided, and multimodally-guided) to construct a large-scale synthetic training dataset with Stable Diffusion. We conduct an extensive comparative study including modality fusion, contrastive learning, reconstruction networks, attention mechanisms, and large vision-language models. Our results show that models trained on image- and multimodally-guided data generalize better to "in the wild" content, due to preserved visual context. However, overall performance remains limited, highlighting the complexity of inferring intent and the need for specialized architectures.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 32 of its 50 references that are in the map. First posted on arXiv 28 Aug 2025.

Builds on

32 of 50 references.

References · 32

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  3. 2020Supervised Contrastive Learningcited by 2,829
  4. 2022Diffusion Models: A Comprehensive Survey of Methods and Applicationscited by 1,114
  5. 2019CNN-generated images are surprisingly easy to spot... for nowcited by 847
  6. 2022On the detection of synthetic images generated by diffusion modelscited by 283
  7. 2020Visual News: Benchmark and Challenges in News Image Captioningcited by 121
  8. 2021NewsCLIPpings: Automatic Generation of Out-of-Context Multimodal Mediacited by 107
  9. 2019Deep Learning for Deepfakes Creation and Detection: A Surveycited by 102
  10. 2022AUC Maximization in the Era of Big Data and AI: A Surveycited by 100
  11. 2020A Survey on Generative Adversarial Networks: Variants, Applications, and Trainingcited by 97
  12. 2024Leveraging Representations from Intermediate Encoder-blocks for Synthetic Image Detectioncited by 89
20 more
  1. 2023CIFAKE: Image Classification and Explainable Identification of AI-Generated Synthetic Imagescited by 84
  2. 2024SNIFFER: Multimodal Large Language Model for Explainable Out-of-Context Misinformation Detectioncited by 74
  3. 2021Open-Domain, Content-based, Multi-modal Fact-checking of Out-of-Context Images via Online Resourcescited by 71
  4. 2024Any-Resolution AI-Generated Image Detection by Spectral Learningcited by 57
  5. 2019r/Fakeddit: A New Multimodal Benchmark Dataset for Fine-grained Fake News Detectioncited by 55
  6. 2023Online Detection of AI-Generated Imagescited by 55
  7. 2023Generalizable Synthetic Image Detection via Language-guided Contrastive Learningcited by 49
  8. 2023VERITE: A Robust Benchmark for Multimodal Misinformation Detection Accounting for Unimodal Biascited by 39
  9. 2023ArtiFact: A Large-Scale Dataset with Artificial and Factual Images for Generalizable and Robust Synthetic Image Detectioncited by 35
  10. 2018Deep Multimodal Image-Repurposing Detectioncited by 31
  11. 2023Deep Image Fingerprint: Towards Low Budget Synthetic Image Detection and Model Lineage Analysiscited by 30
  12. 2023Synthetic Misinformers: Generating and Combating Multimodal Misinformationcited by 23
  13. 2023Support or Refute: Analyzing the Stance of Evidence to Detect Out-of-Context Mis- and Disinformationcited by 21
  14. 2023TWIGMA: A dataset of AI-Generated Images with Metadata From Twittercited by 20
  15. 2023RED-DOT: Multimodal Fact-checking via Relevant Evidence Detectioncited by 18
  16. 2024Similarity over Factuality: Are we making progress on multimodal out-of-context misinformation detection?cited by 11
  17. 2025Navigating the Challenges of AI-Generated Image Detection in the Wild: What Truly Matters?cited by 10
  18. 2024TextureCrop: Enhancing Synthetic Image Detection through Texture-based Croppingcited by 9
  19. 2024Research on Image Super-Resolution Reconstruction Mechanism based on Convolutional Neural Networkcited by 8
  20. 2025Latent Reconstruction from Generated Data for Multimodal Misinformation Detectioncited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026VeriTaS: The First Dynamic Benchmark for Multimodal Automated Fact-Checkingcited by 6
  2. 2026Evidence Triangulation for Multimodal Fact-Checking in the Wildcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024YesBut: A High-Quality Annotated Multimodal Dataset for evaluating Satire Comprehension capability of Vision-Language Models
  2. 2026Beyond the Literal: Decomposing Pragmatic Intent in Multimodal Meme Understanding
  3. 2026The Paradigm Shift: A Comprehensive Survey on Large Vision Language Models for Multimodal Fake News Detection
  4. 2025Towards Unified Multimodal Misinformation Detection in Social Media: A Benchmark Dataset and Baseline
  5. 2026Computational Humor with Multimodal LLMs: Methods, Datasets, Evaluation, and Challenges
  6. 2025Seeing Through Deception: Uncovering Misleading Creator Intent in Multimodal News with Vision-Language Models
  7. 2026Beyond a Joke: Multi-Angle Reasoning for Detecting and Explaining Harmful Humor in Memes
  8. 2026Detecting AI-Generated Content on Social Media with Multi-modal Language Models