# Image Generators are Generalist Vision Learners

arXiv: https://arxiv.org/abs/2604.20329 · HTML: https://mlsanity.com/paper/2604.20329/

## Key facts

- Cited by 43 papers in the map (data through 14 Oct 2026).
- Builds on 95 of its 109 references that are in the map.
- First posted on arXiv 22 Apr 2026.

## Builds on (most cited first, 30 of 95)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities](https://mlsanity.com/paper/2507.06261/) (2025)
- [Finetuned Language Models Are Zero-Shot Learners](https://mlsanity.com/paper/2109.01652/) (2021)
- [Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection](https://mlsanity.com/paper/2303.05499/) (2023)
- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [Sigmoid Loss for Language Image Pre-Training](https://mlsanity.com/paper/2303.15343/) (2023)
- [SAM 2: Segment Anything in Images and Videos](https://mlsanity.com/paper/2408.00714/) (2024)
- [BEiT: BERT Pre-Training of Image Transformers](https://mlsanity.com/paper/2106.08254/) (2021)
- [Improved Baselines with Momentum Contrastive Learning](https://mlsanity.com/paper/2003.04297/) (2020)
- [VGGT: Visual Geometry Grounded Transformer](https://mlsanity.com/paper/2503.11651/) (2025)
- [Depth Anything V2](https://mlsanity.com/paper/2406.09414/) (2024)
- [DINOv3](https://mlsanity.com/paper/2508.10104/) (2025)
- [Depth Map Prediction from a Single Image using a Multi-Scale Deep Network](https://mlsanity.com/paper/1406.2283/) (2014)
- [Learning deep representations by mutual information estimation and maximization](https://mlsanity.com/paper/1808.06670/) (2018)
- [SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Features](https://mlsanity.com/paper/2502.14786/) (2025)
- [Qwen-Image Technical Report](https://mlsanity.com/paper/2508.02324/) (2025)
- [SAM 3: Segment Anything with Concepts](https://mlsanity.com/paper/2511.16719/) (2025)

## Built on by (most cited first, 30 of 43)

- [ImageWAM: Do World Action Models Really Need Video Generation, or Just Image Editing?](https://mlsanity.com/paper/2606.19531/) (2026)
- [VGGT-Omega](https://mlsanity.com/paper/2605.15195/) (2026)
- [Video Generation Models are General-Purpose Vision Learners](https://mlsanity.com/paper/2607.09024/) (2026)
- [Vision as Unified Multimodal Generation](https://mlsanity.com/paper/2607.06560/) (2026)
- [Diffusion Model as a Generalist Segmentation Learner](https://mlsanity.com/paper/2604.24575/) (2026)
- [MARBLE: Multi-Aspect Reward Balance for Diffusion RL](https://mlsanity.com/paper/2605.06507/) (2026)
- [Modality Forcing for Scalable Spatial Generation](https://mlsanity.com/paper/2606.13676/) (2026)
- [SPEAR: A Simulator for Photorealistic Embodied AI Research](https://mlsanity.com/paper/2607.06701/) (2026)
- [PaintBench: Deterministic Evaluation of Precise Visual Editing](https://mlsanity.com/paper/2606.00188/) (2026)
- [iMaC: Translating Actions into Motion and Contact Images for Embodied World Models](https://mlsanity.com/paper/2606.09813/) (2026)
- [Gen-VCoT: Generative Visual Chain-of-Thought Reasoning via Diffusion-Based RGB Intermediate Representations](https://mlsanity.com/paper/2606.16783/) (2026)
- [Show, Don't Tell: Evaluating Spatial Cognition in Generative Pixels Rather Than LLM Text](https://mlsanity.com/paper/2607.21072/) (2026)
- [SUV: Future Scene Understanding as Video Generation for End-to-End Driving](https://mlsanity.com/paper/2608.03084/) (2026)
- [Uncovering Understanding-Generation Synergy in Native Unified Multimodal Models: From Representation, Task to System](https://mlsanity.com/paper/2609.01607/) (2026)
- [Leveraging Image Generators to Address Data Scarcity: The Gen4Regen Dataset for Forest Regeneration Mapping](https://mlsanity.com/paper/2605.05627/) (2026)
- [Thinking with Novel Views: A Systematic Analysis of Generative-Augmented Spatial Intelligence](https://mlsanity.com/paper/2605.10588/) (2026)
- [From RGB Generation to Dense Field Readout: Pixel-Space Dense Prediction with Text-to-Image Models](https://mlsanity.com/paper/2607.06553/) (2026)
- [Image-Space Rule Discovery](https://mlsanity.com/paper/2608.00490/) (2026)
- [Are Image Generators Zero-Shot Perceivers? A Rigorous Evaluation](https://mlsanity.com/paper/2609.07884/) (2026)
- [Marigold V2: Revisiting Diffusion Transformers for Monocular Depth Estimation](https://mlsanity.com/paper/2609.08084/) (2026)
- [The Camera Inside the Editor: Reading the Implicit Camera of Image Editors with Painted Calibration Patterns](https://mlsanity.com/paper/2609.37732/) (2026)
- [PhysEditBench: A Protocol-Conditioned Benchmark for Dense Physical-Map Prediction with Image Editors](https://mlsanity.com/paper/2605.13493/) (2026)
- [Prompting Diffusion Models for Zero-Shot Instance Segmentation](https://mlsanity.com/paper/2606.22660/) (2026)
- [Prompting Image Generators for Training-free Primitive Shape Abstraction](https://mlsanity.com/paper/2607.05568/) (2026)
- [Let RGB Be the Language of Vision](https://mlsanity.com/paper/2607.12450/) (2026)
- [The generator is the tracker: Multi-object tracking by painting persistent identity colours](https://mlsanity.com/paper/2607.17120/) (2026)
- [MODUS: Decoder-Only Any-to-Any Modeling of Diverse Modalities](https://mlsanity.com/paper/2607.25948/) (2026)
- [Hunyuan3D-Buffalo 1.0: A Unified Multimodal Model for Scalable 3D Generation, Understanding, and Editing](https://mlsanity.com/paper/2608.02711/) (2026)
- [MAVISEG: Manifold Propagation and Visual Prototypes for Zero-Shot Open-Vocabulary Segmentation in Diffusion Transformers](https://mlsanity.com/paper/2608.05878/) (2026)
- [From Dense Prediction to Visual Editing: Structured Supervision for Unified Image and Video Creation](https://mlsanity.com/paper/2608.14740/) (2026)

## Abstract (arXiv)

Recent works show that image and video generators exhibit zero-shot visual understanding behaviors, in a way reminiscent of how LLMs develop emergent capabilities of language understanding and reasoning from generative pretraining. While it has long been conjectured that the ability to create visual content implies an ability to understand it, there has been limited evidence that generative vision models have developed strong understanding capabilities. In this work, we demonstrate that image generation training serves a role similar to LLM pretraining, and lets models learn powerful and general visual representations that enable SOTA performance on various vision tasks. We introduce Vision Banana, a generalist model built by instruction-tuning Nano Banana Pro (NBP) on a mixture of its original training data alongside a small amount of vision task data. By parameterizing the output space of vision tasks as RGB images, we seamlessly reframe perception as image generation. Our generalist model, Vision Banana, achieves SOTA results on a variety of vision tasks involving both 2D and 3D understanding, beating or rivaling zero-shot domain-specialists, including Segment Anything Model 3 on segmentation tasks, and the Depth Anything series on metric depth estimation. We show that these results can be achieved with lightweight instruction-tuning without sacrificing the base model's image generation capabilities. The superior results suggest that image generation pretraining is a generalist vision learner. It also shows that image generation serves as a unified and universal interface for vision tasks, similar to text generation's role in language understanding and reasoning. We could be witnessing a major paradigm shift for computer vision, where generative vision pretraining takes a central role in building Foundational Vision Models for both generation and understanding.