DeepArt: A Benchmark to Advance Fidelity Research in AI-Generated Content
On the map: Multimodal LLMs / vision-language models → MLLM training and visual instruction tuning
Abstract (arXiv)
This paper explores the image synthesis capabilities of GPT-4, a leading multi-modal large language model. We establish a benchmark for evaluating the fidelity of texture features in images generated by GPT-4, comprising manually painted pictures and their AI-generated counterparts. The contributions of this study are threefold: First, we provide an in-depth analysis of the fidelity of image synthesis features based on GPT-4, marking the first such study on this state-of-the-art model. Second, the quantitative and qualitative experiments fully reveals the limitations of the GPT-4 model in image synthesis. Third, we have compiled a unique benchmark of manual drawings and corresponding GPT-4-generated images, introducing a new task to advance fidelity research in AI-generated content (AIGC). The dataset is available at: urlhttps://github.com/rickwang28574/DeepArt.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in MLLM training and visual instruction tuning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 20 of its 33 references that are in the map. First posted on arXiv 16 Dec 2023.
Builds on
20 of 33 references.
References · 20
- 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,711
- 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,592
- 2023Capabilities of GPT-4 on Medical Challenge Problemscited by 638
- 2021Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Surveycited by 463
- 2023Segment Anything Model for Medical Image Analysis: an Experimental Studycited by 291
- 2023Multimodal Foundation Models: From Specialists to General-Purpose Assistantscited by 247
- 2023A Comprehensive Survey of AI-Generated Content (AIGC): A History of Generative AI from GAN to ChatGPTcited by 224
- 2023Evaluating the Logical Reasoning Ability of ChatGPT and GPT-4cited by 202
- 2023Continual Pre-Training of Large Language Models: How to (re)warm your model?cited by 137
- 2021An Empirical Investigation of the Role of Pre-training in Lifelong Learningcited by 134
- 2023A Complete Survey on Generative AI (AIGC): Is ChatGPT from GPT-4 to GPT-5 All You Need?cited by 84
- 2023GPTEval: A Survey on Assessments of ChatGPT and GPT-4cited by 73
8 more
- 2023Promptify: Text-to-Image Generation through Interactive Prompt Exploration with Large Language Modelscited by 67
- 2022Large-scale Text-to-Image Generation Models for Visual Artists' Creative Workscited by 61
- 2023AI-Generated Content (AIGC): A Surveycited by 59
- 2023An Early Evaluation of GPT-4V(ision)cited by 31
- 2021Paint4Poem: A Dataset for Artistic Visualization of Classical Chinese Poemscited by 18
- 2019Model-Based and Data-Driven Strategies in Medical Image Computingcited by 16
- 2023Generative AI for Product Design: Getting the Right Design and the Design Rightcited by 6
- 2023Beyond Segmentation: Road Network Generation with Multi-Modal LLMscited by 3
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2024GPTDrawer: Enhancing Visual Synthesis through ChatGPT
- 2025An Empirical Study of GPT-4o Image Generation Capabilities
- 2025GPT-ImgEval: A Comprehensive Benchmark for Diagnosing GPT4o in Image Generation
- 2025Style Transfer: A Decade Survey
- 2024TypeScore: A Text Fidelity Metric for Text-to-Image Generative Models
- 2025ShareGPT-4o-Image: Aligning Multimodal Models with GPT-4o-Level Image Generation
- 2025Why Compress What You Can Generate? When GPT-4o Generation Ushers in Image Compression Fields
- 2026Graphic-Design-Bench: A Comprehensive Benchmark for Evaluating AI on Graphic Design Tasks