# "Humor, Art, or Misinformation?": A Multimodal Dataset for Intent-Aware Synthetic Image Detection

arXiv: https://arxiv.org/abs/2508.20670 · HTML: https://mlsanity.com/paper/2508.20670/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 50 references that are in the map.
- First posted on arXiv 28 Aug 2025.

## Builds on (most cited first, 30 of 32)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Supervised Contrastive Learning](https://mlsanity.com/paper/2004.11362/) (2020)
- [Diffusion Models: A Comprehensive Survey of Methods and Applications](https://mlsanity.com/paper/2209.00796/) (2022)
- [CNN-generated images are surprisingly easy to spot... for now](https://mlsanity.com/paper/1912.11035/) (2019)
- [On the detection of synthetic images generated by diffusion models](https://mlsanity.com/paper/2211.00680/) (2022)
- [Visual News: Benchmark and Challenges in News Image Captioning](https://mlsanity.com/paper/2010.03743/) (2020)
- [NewsCLIPpings: Automatic Generation of Out-of-Context Multimodal Media](https://mlsanity.com/paper/2104.05893/) (2021)
- [Deep Learning for Deepfakes Creation and Detection: A Survey](https://mlsanity.com/paper/1909.11573/) (2019)
- [AUC Maximization in the Era of Big Data and AI: A Survey](https://mlsanity.com/paper/2203.15046/) (2022)
- [A Survey on Generative Adversarial Networks: Variants, Applications, and Training](https://mlsanity.com/paper/2006.05132/) (2020)
- [Leveraging Representations from Intermediate Encoder-blocks for Synthetic Image Detection](https://mlsanity.com/paper/2402.19091/) (2024)
- [CIFAKE: Image Classification and Explainable Identification of AI-Generated Synthetic Images](https://mlsanity.com/paper/2303.14126/) (2023)
- [SNIFFER: Multimodal Large Language Model for Explainable Out-of-Context Misinformation Detection](https://mlsanity.com/paper/2403.03170/) (2024)
- [Open-Domain, Content-based, Multi-modal Fact-checking of Out-of-Context Images via Online Resources](https://mlsanity.com/paper/2112.00061/) (2021)
- [Any-Resolution AI-Generated Image Detection by Spectral Learning](https://mlsanity.com/paper/2411.19417/) (2024)
- [r/Fakeddit: A New Multimodal Benchmark Dataset for Fine-grained Fake News Detection](https://mlsanity.com/paper/1911.03854/) (2019)
- [Online Detection of AI-Generated Images](https://mlsanity.com/paper/2310.15150/) (2023)
- [Generalizable Synthetic Image Detection via Language-guided Contrastive Learning](https://mlsanity.com/paper/2305.13800/) (2023)
- [VERITE: A Robust Benchmark for Multimodal Misinformation Detection Accounting for Unimodal Bias](https://mlsanity.com/paper/2304.14133/) (2023)
- [ArtiFact: A Large-Scale Dataset with Artificial and Factual Images for Generalizable and Robust Synthetic Image Detection](https://mlsanity.com/paper/2302.11970/) (2023)
- [Deep Multimodal Image-Repurposing Detection](https://mlsanity.com/paper/1808.06686/) (2018)
- [Deep Image Fingerprint: Towards Low Budget Synthetic Image Detection and Model Lineage Analysis](https://mlsanity.com/paper/2303.10762/) (2023)
- [Synthetic Misinformers: Generating and Combating Multimodal Misinformation](https://mlsanity.com/paper/2303.01217/) (2023)
- [Support or Refute: Analyzing the Stance of Evidence to Detect Out-of-Context Mis- and Disinformation](https://mlsanity.com/paper/2311.01766/) (2023)
- [TWIGMA: A dataset of AI-Generated Images with Metadata From Twitter](https://mlsanity.com/paper/2306.08310/) (2023)
- [RED-DOT: Multimodal Fact-checking via Relevant Evidence Detection](https://mlsanity.com/paper/2311.09939/) (2023)
- [Similarity over Factuality: Are we making progress on multimodal out-of-context misinformation detection?](https://mlsanity.com/paper/2407.13488/) (2024)
- [Navigating the Challenges of AI-Generated Image Detection in the Wild: What Truly Matters?](https://mlsanity.com/paper/2507.10236/) (2025)
- [TextureCrop: Enhancing Synthetic Image Detection through Texture-based Cropping](https://mlsanity.com/paper/2407.15500/) (2024)

## Built on by (most cited first, 2 of 2)

- [VeriTaS: The First Dynamic Benchmark for Multimodal Automated Fact-Checking](https://mlsanity.com/paper/2601.08611/) (2026)
- [Evidence Triangulation for Multimodal Fact-Checking in the Wild](https://mlsanity.com/paper/2606.31367/) (2026)

## Abstract (arXiv)

Recent advances in multimodal AI have enabled progress in detecting synthetic and out-of-context content. However, existing efforts largely overlook the intent behind AI-generated images. To fill this gap, we introduce S-HArM, a multimodal dataset for intent-aware classification, comprising 9,576 "in the wild" image-text pairs from Twitter/X and Reddit, labeled as Humor/Satire, Art, or Misinformation. Additionally, we explore three prompting strategies (image-guided, description-guided, and multimodally-guided) to construct a large-scale synthetic training dataset with Stable Diffusion. We conduct an extensive comparative study including modality fusion, contrastive learning, reconstruction networks, attention mechanisms, and large vision-language models. Our results show that models trained on image- and multimodally-guided data generalize better to "in the wild" content, due to preserved visual context. However, overall performance remains limited, highlighting the complexity of inferring intent and the need for specialized architectures.