# "That's another doom I haven't thought about": A User Study on AI Labels as a Safeguard Against Image-Based Misinformation

arXiv: https://arxiv.org/abs/2505.22845 · HTML: https://mlsanity.com/paper/2505.22845/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 149 references that are in the map.
- First posted on arXiv 28 May 2025.

## Builds on (most cited first, 20 of 20)

- [CNN-generated images are surprisingly easy to spot... for now](https://mlsanity.com/paper/1912.11035/) (2019)
- [Towards Universal Fake Image Detectors that Generalize Across Generative Models](https://mlsanity.com/paper/2302.10174/) (2023)
- [HiDDeN: Hiding Data With Deep Networks](https://mlsanity.com/paper/1807.09937/) (2018)
- [StegaStamp: Invisible Hyperlinks in Physical Photographs](https://mlsanity.com/paper/1904.05343/) (2019)
- [Gaussian Shading: Provable Performance-Lossless Image Watermarking for Diffusion Models](https://mlsanity.com/paper/2404.04956/) (2024)
- [AEROBLADE: Training-Free Detection of Latent Diffusion Images Using Autoencoder Reconstruction Error](https://mlsanity.com/paper/2401.17879/) (2024)
- ['It's Reducing a Human Being to a Percentage'; Perceptions of Justice in Algorithmic Decisions](https://mlsanity.com/paper/1801.10408/) (2018)
- [RingID: Rethinking Tree-Ring Watermarking for Enhanced Multi-Key Identification](https://mlsanity.com/paper/2404.14055/) (2024)
- [Robustness of AI-Image Detectors: Fundamental Limits and Practical Attacks](https://mlsanity.com/paper/2310.00076/) (2023)
- [A Systematic Literature Review of User Trust in AI-Enabled Systems: An HCI Perspective](https://mlsanity.com/paper/2304.08795/) (2023)
- [Using AI Assistants in Software Development: A Qualitative Study on Security Practices and Concerns](https://mlsanity.com/paper/2405.06371/) (2024)
- [AMMeBa: A Large-Scale Survey and Dataset of Media-Based Misinformation In-The-Wild](https://mlsanity.com/paper/2405.11697/) (2024)
- [A Representative Study on Human Detection of Artificially Generated Media Across Countries](https://mlsanity.com/paper/2312.05976/) (2023)
- [An Analysis of Recent Advances in Deepfake Image Detection in an Evolving Threat Landscape](https://mlsanity.com/paper/2404.16212/) (2024)
- ["Look! It's a Computer Program! It's an Algorithm! It's AI!": Does Terminology Affect Human Perceptions and Evaluations of Algorithmic Decision-Making Systems?](https://mlsanity.com/paper/2108.11486/) (2021)
- [Examining the Impact of Provenance-Enabled Media on Trust and Accuracy Perceptions](https://mlsanity.com/paper/2303.12118/) (2023)
- [The effect of source disclosure on evaluation of AI-generated messages: A two-part study](https://mlsanity.com/paper/2311.15544/) (2023)
- [Adapting Security Warnings to Counter Online Disinformation](https://mlsanity.com/paper/2008.10772/) (2020)
- [Adversarial Robustness of AI-Generated Image Detectors in the Real World](https://mlsanity.com/paper/2410.01574/) (2024)
- [Signals of Provenance: Practices & Challenges of Navigating Indicators in AI-Generated Media for Sighted and Blind Individuals](https://mlsanity.com/paper/2505.16057/) (2025)

## Built on by (most cited first, 3 of 3)

- [First-Place Solution to NeurIPS 2024 Invisible Watermark Removal Challenge](https://mlsanity.com/paper/2508.21072/) (2025)
- [RAVEN: Erasing Invisible Watermarks via Novel View Synthesis](https://mlsanity.com/paper/2601.08832/) (2026)
- [Understanding Security and Privacy Perceptions of Content Creators Regarding AI Labels of AI-Generated Content](https://mlsanity.com/paper/2608.08129/) (2026)

## Abstract (arXiv)

As generative AI is increasingly contributing to the spread of deceptively realistic misinformation, lawmakers have introduced regulations requiring the disclosure of AI-generated content. However, it is unclear if labels reduce the risk of users falling for AI-generated misinformation. To address this research gap, we study the effect of labels on users' perception and the implications of mislabeling, focusing on AI-generated images. We first explored users' opinions and expectations of labels using five focus groups. Although participants were wary of practical implementations, they considered labeling helpful in identifying AI-generated images and avoiding deception. Second, we conducted a survey with 1354 participants to assess how labels affect users' ability to recognize misinformation. While labels reduced participants' belief in false claims supported by AI-generated images, we found evidence of overreliance, leading to unintended side effects: Participants were more susceptible to false claims accompanied by human-made images, and were more hesitant to believe true claims illustrated with labeled AI-generated images.