# Creation-MMBench: Assessing Context-Aware Creative Intelligence in MLLM

arXiv: https://arxiv.org/abs/2503.14478 · HTML: https://mlsanity.com/paper/2503.14478/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 36 references that are in the map.
- First posted on arXiv 18 Mar 2025.

## Builds on (most cited first, 23 of 23)

- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [Measuring Mathematical Problem Solving With the MATH Dataset](https://mlsanity.com/paper/2103.03874/) (2021)
- [GPQA: A Graduate-Level Google-Proof Q&A Benchmark](https://mlsanity.com/paper/2311.12022/) (2023)
- [InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks](https://mlsanity.com/paper/2312.14238/) (2023)
- [MMBench: Is Your Multi-modal Model an All-around Player?](https://mlsanity.com/paper/2307.06281/) (2023)
- [MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI](https://mlsanity.com/paper/2311.16502/) (2023)
- [MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmark](https://mlsanity.com/paper/2406.01574/) (2024)
- [MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts](https://mlsanity.com/paper/2310.02255/) (2023)
- [MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilities](https://mlsanity.com/paper/2308.02490/) (2023)
- [Are We on the Right Way for Evaluating Large Vision-Language Models?](https://mlsanity.com/paper/2403.20330/) (2024)
- [VLMEvalKit: An Open-Source Toolkit for Evaluating Large Multi-Modality Models](https://mlsanity.com/paper/2407.11691/) (2024)
- [EmbodiedBench: Comprehensive Benchmarking Multi-modal Large Language Models for Vision-Driven Embodied Agents](https://mlsanity.com/paper/2502.09560/) (2025)
- [VLABench: A Large-Scale Benchmark for Language-Conditioned Robotics Manipulation with Long-Horizon Reasoning Tasks](https://mlsanity.com/paper/2412.18194/) (2024)
- [Can MLLMs Reason in Multimodality? EMMA: An Enhanced MultiModal ReAsoning Benchmark](https://mlsanity.com/paper/2501.05444/) (2025)
- [A Confederacy of Models: a Comprehensive Evaluation of LLMs on Creative Writing](https://mlsanity.com/paper/2310.08433/) (2023)
- [Putting GPT-3's Creativity to the (Alternative Uses) Test](https://mlsanity.com/paper/2206.08932/) (2022)
- [Prism: A Framework for Decoupling and Assessing the Capabilities of VLMs](https://mlsanity.com/paper/2406.14544/) (2024)
- [MLLM-Bench: Evaluating Multimodal LLMs with Per-sample Criteria](https://mlsanity.com/paper/2311.13951/) (2023)
- [Redundancy Principles for MLLMs Benchmarks](https://mlsanity.com/paper/2501.13953/) (2025)
- [RPGBENCH: Evaluating Large Language Models as Role-Playing Game Engines](https://mlsanity.com/paper/2502.00595/) (2025)
- [AlignMMBench: Evaluating Chinese Multimodal Alignment in Large Vision-Language Models](https://mlsanity.com/paper/2406.09295/) (2024)
- [SimulBench: Evaluating Language Models with Creative Simulation Tasks](https://mlsanity.com/paper/2409.07641/) (2024)
- [Do LLMs Agree on the Creativity Evaluation of Alternative Uses?](https://mlsanity.com/paper/2411.15560/) (2024)

## Built on by (most cited first, 14 of 14)

- [MM-IFEngine: Towards Multimodal Instruction Following](https://mlsanity.com/paper/2504.07957/) (2025)
- [What Shapes a Creative Machine Mind? Comprehensively Benchmarking Creativity in Foundation Models](https://mlsanity.com/paper/2510.04009/) (2025)
- [Children's Intelligence Tests Pose Challenges for MLLMs? KidGym: A 2D Grid-Based Reasoning Benchmark for MLLMs](https://mlsanity.com/paper/2603.20209/) (2026)
- [GuessBench: Sensemaking Multimodal Creativity in the Wild](https://mlsanity.com/paper/2506.00814/) (2025)
- [Bi-Level Prompt Optimization for Multimodal LLM-as-a-Judge](https://mlsanity.com/paper/2602.11340/) (2026)
- [PosterIQ: A Design Perspective Benchmark for Poster Understanding and Generation](https://mlsanity.com/paper/2603.24078/) (2026)
- [TeamLLM: A Human-Like Team-Oriented Collaboration Framework for Multi-Step Contextualized Tasks](https://mlsanity.com/paper/2604.06765/) (2026)
- [AlphaContext: An Evolutionary Tree-based Psychometric Context Generator for Creativity Assessment](https://mlsanity.com/paper/2604.18398/) (2026)
- [QoNext: Towards Next-generation QoE for Foundation Models](https://mlsanity.com/paper/2509.21889/) (2025)
- [Beyond the Current Observation: Evaluating Multimodal Large Language Models in Controllable Non-Markov Games](https://mlsanity.com/paper/2606.19338/) (2026)
- [AGC-Bench: Measuring Artificial General Creativity](https://mlsanity.com/paper/2607.01152/) (2026)
- [Can MLLMs Decode the Creative Leap? Introducing C4 for Cross-Concept Understanding](https://mlsanity.com/paper/2608.06501/) (2026)
- [Can Language Models Imagine Without Seeing? Ekphrasis: Measuring Visual Creative Ideation in Text-Only LLMs](https://mlsanity.com/paper/2608.06967/) (2026)
- [Repurposing Pre-trained LLMs as High Fidelity Continuous Text Autoencoders](https://mlsanity.com/paper/2609.27248/) (2026)

## Abstract (arXiv)

Creativity is a fundamental aspect of intelligence, involving the ability to generate novel and appropriate solutions across diverse contexts. While Large Language Models (LLMs) have been extensively evaluated for their creative capabilities, the assessment of Multimodal Large Language Models (MLLMs) in this domain remains largely unexplored. To address this gap, we introduce Creation-MMBench, a multimodal benchmark specifically designed to evaluate the creative capabilities of MLLMs in real-world, image-based tasks. The benchmark comprises 765 test cases spanning 51 fine-grained tasks. To ensure rigorous evaluation, we define instance-specific evaluation criteria for each test case, guiding the assessment of both general response quality and factual consistency with visual inputs. Experimental results reveal that current open-source MLLMs significantly underperform compared to proprietary models in creative tasks. Furthermore, our analysis demonstrates that visual fine-tuning can negatively impact the base LLM's creative abilities. Creation-MMBench provides valuable insights for advancing MLLM creativity and establishes a foundation for future improvements in multimodal generative intelligence. Full data and evaluation code is released on https://github.com/open-compass/Creation-MMBench.