# PixelArena: A benchmark for Pixel-Precision Visual Intelligence

arXiv: https://arxiv.org/abs/2512.16303 · HTML: https://mlsanity.com/paper/2512.16303/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 37 references that are in the map.
- First posted on arXiv 18 Dec 2025.

## Builds on (most cited first, 26 of 26)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Masked-attention Mask Transformer for Universal Image Segmentation](https://mlsanity.com/paper/2112.01527/) (2021)
- [Scaling Autoregressive Models for Content-Rich Text-to-Image Generation](https://mlsanity.com/paper/2206.10789/) (2022)
- [SAM 3: Segment Anything with Concepts](https://mlsanity.com/paper/2511.16719/) (2025)
- [MaskGAN: Towards Diverse and Interactive Facial Image Manipulation](https://mlsanity.com/paper/1907.11922/) (2019)
- [MagicBrush: A Manually Annotated Dataset for Instruction-Guided Image Editing](https://mlsanity.com/paper/2306.10012/) (2023)
- [ELLA: Equip Diffusion Models with LLM for Enhanced Semantic Alignment](https://mlsanity.com/paper/2403.05135/) (2024)
- [Generative Multimodal Models are In-Context Learners](https://mlsanity.com/paper/2312.13286/) (2023)
- [TIFA: Accurate and Interpretable Text-to-Image Faithfulness Evaluation with Question Answering](https://mlsanity.com/paper/2303.11897/) (2023)
- [Mechanistic Interpretability for AI Safety -- A Review](https://mlsanity.com/paper/2404.14082/) (2024)
- [OneFormer: One Transformer to Rule Universal Image Segmentation](https://mlsanity.com/paper/2211.06220/) (2022)
- [Rethinking FID: Towards a Better Evaluation Metric for Image Generation](https://mlsanity.com/paper/2401.09603/) (2023)
- [T2I-CompBench++: An Enhanced and Comprehensive Benchmark for Compositional Text-to-image Generation](https://mlsanity.com/paper/2307.06350/) (2023)
- [ImgEdit: A Unified Image Editing Dataset and Benchmark](https://mlsanity.com/paper/2505.20275/) (2025)
- [General Facial Representation Learning in a Visual-Linguistic Manner](https://mlsanity.com/paper/2112.03109/) (2021)
- [FSS-1000: A 1000-Class Dataset for Few-Shot Segmentation](https://mlsanity.com/paper/1907.12347/) (2019)
- [Holistic Evaluation of Text-To-Image Models](https://mlsanity.com/paper/2311.04287/) (2023)
- [FakeShield: Explainable Image Forgery Detection and Localization via Multi-modal Large Language Models](https://mlsanity.com/paper/2410.02761/) (2024)
- [ImagenHub: Standardizing the evaluation of conditional image generation models](https://mlsanity.com/paper/2310.01596/) (2023)
- [SAM4MLLM: Enhance Multi-Modal Large Language Model for Referring Expression Segmentation](https://mlsanity.com/paper/2409.10542/) (2024)
- [Uni-MoE-2.0-Omni: Scaling Language-Centric Omnimodal Large Model with Advanced MoE, Training and Data](https://mlsanity.com/paper/2511.12609/) (2025)
- [Roboflow100-VL: A Multi-Domain Object Detection Benchmark for Vision-Language Models](https://mlsanity.com/paper/2505.20612/) (2025)
- [SegFace: Face Segmentation of Long-Tail Classes](https://mlsanity.com/paper/2412.08647/) (2024)
- [Refer to Any Segmentation Mask Group With Vision-Language Prompts](https://mlsanity.com/paper/2506.05342/) (2025)
- [DitHub: A Modular Framework for Incremental Open-Vocabulary Object Detection](https://mlsanity.com/paper/2503.09271/) (2025)

## Built on by (most cited first, 1 of 1)

- [SegBanana: Steering Unified Multimodal Models into Medical Segmenters](https://mlsanity.com/paper/2609.34235/) (2026)

## Abstract (arXiv)

Omni-modal models that have multimodal input and output are emerging. However, benchmarking their multimodal generation, especially in image generation, is challenging due to the subtleties of human preferences and model biases. Many image generation benchmarks focus on aesthetics instead of the fine-grained generation capabilities of these models, failing to evaluate their visual intelligence with objective metrics. In PixelArena, we propose using semantic segmentation tasks to objectively examine their fine-grained generative intelligence with pixel precision. With our benchmark and experiments, we find the latest Gemini 3 Pro Image has emergent image generation capabilities that generate semantic masks with high fidelity under zero-shot settings, showcasing visual intelligence unseen before and true generalization in new image generation tasks. We further investigate its results, compare them qualitatively and quantitatively with those of other models, and present failure cases. The findings not only signal exciting progress in the field but also provide insights into future research related to dataset development, omni-modal model development, and the design of metrics.