# Edited Media Understanding Frames: Reasoning About the Intent and Implications of Visual Misinformation

arXiv: https://arxiv.org/abs/2012.04726 · HTML: https://mlsanity.com/paper/2012.04726/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Dec 2020.

## Built on by (most cited first, 15 of 15)

- [Teach Me to Explain: A Review of Datasets for Explainable Natural Language Processing](https://mlsanity.com/paper/2102.12060/) (2021)
- [MMFakeBench: A Mixed-Source Multimodal Misinformation Detection Benchmark for LVLMs](https://mlsanity.com/paper/2406.08772/) (2024)
- [Local Interpretations for Explainable Natural Language Processing: A Survey](https://mlsanity.com/paper/2103.11072/) (2021)
- [Matching Tweets With Applicable Fact-Checks Across Languages](https://mlsanity.com/paper/2202.07094/) (2022)
- [MFC-Bench: Benchmarking Multimodal Fact-Checking with Large Vision-Language Models](https://mlsanity.com/paper/2406.11288/) (2024)
- [Combating Online Misinformation Videos: Characterization, Detection, and Future Directions](https://mlsanity.com/paper/2302.03242/) (2023)
- [MVP-Bench: Can Large Vision--Language Models Conduct Multi-level Visual Perception Like Humans?](https://mlsanity.com/paper/2410.04345/) (2024)
- [M4FC: a Multimodal, Multilingual, Multicultural, Multitask Real-World Fact-Checking Dataset](https://mlsanity.com/paper/2510.23508/) (2025)
- [Seeing Through Deception: Uncovering Misleading Creator Intent in Multimodal News with Vision-Language Models](https://mlsanity.com/paper/2505.15489/) (2025)
- ["Image, Tell me your story!" Predicting the original meta-context of visual misinformation](https://mlsanity.com/paper/2408.09939/) (2024)
- [The Psychology of Falsehood: A Human-Centric Survey of Misinformation Detection](https://mlsanity.com/paper/2509.15896/) (2025)
- [Towards Explainable Bilingual Multimodal Misinformation Detection and Localization](https://mlsanity.com/paper/2506.22930/) (2025)
- [Harmful Visual Content Manipulation Matters in Misinformation Detection Under Multimedia Scenarios](https://mlsanity.com/paper/2603.21054/) (2026)
- [ReMMD: Realistic Multilingual Multi-Image Agentic Verification for Multimodal Misinformation Detection](https://mlsanity.com/paper/2606.24112/) (2026)
- [RCMN: Understanding Misleadingness in Influential Public Discourse](https://mlsanity.com/paper/2608.27358/) (2026)

## Abstract (arXiv)

Multimodal disinformation, from 'deepfakes' to simple edits that deceive, is an important societal problem. Yet at the same time, the vast majority of media edits are harmless -- such as a filtered vacation photo. The difference between this example, and harmful edits that spread disinformation, is one of intent. Recognizing and describing this intent is a major challenge for today's AI systems. We present the task of Edited Media Understanding, requiring models to answer open-ended questions that capture the intent and implications of an image edit. We introduce a dataset for our task, EMU, with 48k question-answer pairs written in rich natural language. We evaluate a wide variety of vision-and-language models for our task, and introduce a new model PELICAN, which builds upon recent progress in pretrained multimodal representations. Our model obtains promising results on our dataset, with humans rating its answers as accurate 40.35% of the time. At the same time, there is still much work to be done -- humans prefer human-annotated captions 93.56% of the time -- and we provide analysis that highlights areas for further progress.