mlsanityData to 14 Oct 2026

Vision as Unified Multimodal Generation

arXiv 2607.06560 ↗First posted 7 Jul 2026cs.CVcited by 7

Not on the map. Why?

Abstract (arXiv)

We formulate computer vision as unified multimodal generation, where heterogeneous visual tasks are expressed in the native text and image generation spaces of a unified multimodal model, without task-specific architectures. Under this formulation, SenseNova-Vision uses natural-language instructions and optional visual prompts to specify tasks, target regions or views, and decoding conventions, and generates responses as text for symbolic outputs, images for dense spatial predictions, or mixed text-and-image outputs for compositional tasks. To support large-scale training, we convert diverse computer vision annotations into instruction-response examples compatible with these generation spaces, resulting in the SenseNova-Vision Corpus, a computer-vision instruction-response corpus spanning text, image, and mixed targets. Starting from an off-the-shelf pretrained unified multimodal model, SenseNova-Vision is trained primarily on this corpus, with auxiliary multimodal data used as a capability-preserving mixture, and requires no task-specific prediction heads or architectural modifications. The resulting model covers a broad range of vision tasks, including detection, OCR, keypoint estimation, segmentation, depth estimation, surface normal prediction, point maps, and camera pose estimation, while supporting language-defined variants that combine category, color, region, and other visual cues. Experiments show that a single unified model can match leading task-specialized systems across structured visual understanding, dense geometric prediction, segmentation, and multi-view visual geometry. These results suggest unified multimodal generation as a scalable route for integrating computer vision capabilities into general-purpose foundation models. The model and corpus are publicly available.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 171 of its 215 references that are in the map. First posted on arXiv 7 Jul 2026.

Builds on

171 of 215 references.

References · 171

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  5. 2023Segment Anythingcited by 8,234
  6. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  7. 2023Visual Instruction Tuningcited by 7,869
  8. 2020End-to-End Object Detection with Transformerscited by 7,414
  9. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  10. 2017Mask R-CNNcited by 7,277
  11. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,444
  12. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
38 more
  1. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
  2. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,117
  3. 2025Qwen3-VL Technical Reportcited by 2,793
  4. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
  5. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,415
  6. 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,351
  7. 2023DUSt3R: Geometric 3D Vision Made Easycited by 1,567
  8. 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,540
  9. 2025VGGT: Visual Geometry Grounded Transformercited by 1,539
  10. 2024Depth Anything V2cited by 1,422
  11. 2015Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Modelscited by 1,362
  12. 2024Depth Anything: Unleashing the Power of Large-Scale Unlabeled Datacited by 1,360
  13. 2018BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learningcited by 1,288
  14. 2019LVIS: A Dataset for Large Vocabulary Instance Segmentationcited by 1,217
  15. 2025SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Featurescited by 1,188
  16. 2023Kosmos-2: Grounding Multimodal Large Language Models to the Worldcited by 1,089
  17. 2016Modeling Context in Referring Expressionscited by 1,063
  18. 2023GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignmentcited by 972
  19. 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 927
  20. 2023Shikra: Unleashing Multimodal LLM's Referential Dialogue Magiccited by 906
  21. 2025Emerging Properties in Unified Multimodal Pretrainingcited by 897
  22. 2023LISA: Reasoning Segmentation via Large Language Modelcited by 776
  23. 2023ScanNet++: A High-Fidelity Dataset of 3D Indoor Scenescited by 764
  24. 2024Show-o: One Single Transformer to Unify Multimodal Understanding and Generationcited by 752
  25. 2022OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning Frameworkcited by 745
  26. 2024Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMscited by 735
  27. 2025Depth Anything 3: Recovering the Visual Space from Any Viewscited by 692
  28. 2021Common Objects in 3D: Large-Scale Learning and Evaluation of Real-life 3D Category Reconstructioncited by 652
  29. 2018Taskonomy: Disentangling Task Transfer Learningcited by 634
  30. 2020Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understandingcited by 604
  31. 2023DL3DV-10K: A Large-Scale Scene Dataset for Deep Learning-based 3D Visioncited by 537
  32. 2023Ferret: Refer and Ground Anything Anywhere at Any Granularitycited by 509
  33. 2020TartanAir: A Dataset to Push the Limits of Visual SLAMcited by 506
  34. 2019BlendedMVS: A Large-scale Dataset for Generalized Multi-view Stereo Networkscited by 490
  35. 2019Scaling Out-of-Distribution Detection for Real-World Settingscited by 465
  36. 2024Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and Generationcited by 457
  37. 2023GLaMM: Pixel Grounding Large Multimodal Modelcited by 455
  38. 2022Unified-IO: A Unified Model for Vision, Language, and Multi-Modal Taskscited by 421

50 most cited of 171 All 171 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 7.

7 papers.