Can You Trust What You See? Human and AI Detection of Synthetic Legal Evidence
Not on the map. Why?
Abstract (arXiv)
Visual evidence has long been treated as a reliable form of legal proof, but advances in artificial intelligence (AI) are undermining that assumption. This article asks how well humans and frontier multimodal large language models (MLLMs) can distinguish authentic evidentiary photographs from AI-generated counterparts in the object-centric scenarios typical of civil disputes. We built Synthetic Legal Evidence Detection (SLED-1400), a dataset of 200 authentic evidence images paired with 1,200 synthetic counterparts produced by six contemporary text-to-image generators across ten evidence categories. The same stimuli and response format were used in a controlled web experiment with 136 lay participants and in a standardized evaluation of four MLLMs (GPT-5.1, Gemini-3-Pro, Gemini-3-Flash, Qwen3-VL-235B). Human accuracy was 64.8% overall, and 48.5% and 51.0% on the two strongest generators (Gemini-3-Pro-Image and Flux-2-Max), indistinguishable from chance. MLLMs never misclassified an authentic image (100% specificity), but missed most synthetic outputs from the harder generators, with average MLLM detection at 5.9% on Gemini-3-Pro-Image outputs. Human and MLLM errors were largely uncorrelated, while the four MLLMs were strongly correlated with each other. Neither group is a reliable standalone authenticator. We argue that visual evidence in legal proceedings should be treated as inherently contestable, and that a workable procedural response must combine trained human review, MLLM screening, and provenance infrastructure such as C2PA Content Credentials.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 21 of its 52 references that are in the map. First posted on arXiv 29 May 2026.
Builds on
21 of 52 references.
References · 21
- 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
- 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
- 2024Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preferencecited by 1,151
- 2019FaceForensics++: Learning to Detect Manipulated Facial Imagescited by 1,095
- 2019CNN-generated images are surprisingly easy to spot... for nowcited by 842
- 2023Vision-Language Models for Vision Tasks: A Surveycited by 834
- 2022Exploring Plain Vision Transformer Backbones for Object Detectioncited by 723
- 2023Towards Universal Fake Image Detectors that Generalize Across Generative Modelscited by 483
- 2023Raising the Bar of AI-generated Image Detection with CLIPcited by 163
- 2023Online Detection of AI-Generated Imagescited by 55
- 2025SynthID-Image: Image watermarking at internet scalecited by 26
- 2021More Real than Real: A Study on Human Visual Perception of Synthetic Facescited by 23
9 more
- 2023A Representative Study on Human Detection of Artificially Generated Media Across Countriescited by 22
- 2021A Study of the Human Perception of Synthetic Facescited by 14
- 2024Robots in the Middle: Evaluating LLMs in Dispute Resolutioncited by 13
- 2025Methods and Trends in Detecting AI-Generated Images: A Comprehensive Reviewcited by 12
- 2025VER-Bench: Evaluating MLLMs on Reasoning with Fine-Grained Visual Evidencecited by 9
- 2025OpenFake: An Open Dataset and Platform Toward Real-World Deepfake Detectioncited by 9
- 2024Analyzing Images of Legal Documents: Toward Multi-Modal LLMs for Access to Justicecited by 3
- 2025Can LLMs Create Legally Relevant Summaries and Analyses of Videos?cited by 1
- 2025LegalWebAgent: Empowering Access to Justice via LLM-Based Web Agentscited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026AI-generated Images Challenge Visual Trust in High-risk Scenarios
- 2025Survey on AI-Generated Media Detection: From Non-MLLM to MLLM
- 2026Text Over Image: Auditing Multimodal Robustness in Synthetic Medical Image Detection
- 2024FakeBench: Probing Explainable Fake Image Detection via Large Multimodal Models
- 2025Spot the Fake: Large Multimodal Model-Based Synthetic Image Detection with Artifact Explanation
- 2025FakeScope: Large Multimodal Expert Model for Transparent AI-Generated Image Forensics
- 2025Could AI Trace and Explain the Origins of AI-Generated Images and Text?
- 2025Interpretable and Reliable Detection of AI-Generated Images via Grounded Reasoning in MLLMs