mlsanityData to 14 Oct 2026

Omni-View: Unlocking How Generation Facilitates Understanding in Unified 3D Model based on Multiview images

arXiv 2511.07222 ↗First posted 10 Nov 2025cs.CVcited by 17

Not on the map. Why?

Abstract (arXiv)

This paper presents Omni-View, which extends the unified multimodal understanding and generation to 3D scenes based on multiview images, exploring the principle that "generation facilitates understanding". Consisting of understanding model, texture module, and geometry module, Omni-View jointly models scene understanding, novel view synthesis, and geometry estimation, enabling synergistic interaction between 3D scene understanding and generation tasks. By design, it leverages the spatiotemporal modeling capabilities of its texture module responsible for appearance synthesis, alongside the explicit geometric constraints provided by its dedicated geometry module, thereby enriching the model's holistic understanding of 3D scenes. Trained with a two-stage strategy, Omni-View achieves a state-of-the-art score of 55.4 on the VSI-Bench benchmark, outperforming existing specialized 3D understanding models, while simultaneously delivering strong performance in both novel view synthesis and 3D scene generation. The code and pretraiend models are open-sourced at https://github.com/AIDC-AI/Omni-View.

Key facts. Cited by 17 papers in the map (data through 14 Oct 2026). Builds on 51 of its 63 references that are in the map. First posted on arXiv 10 Nov 2025.

Builds on

51 of 63 references.

References · 51

  1. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  2. 2022Flow Matching for Generative Modelingcited by 5,702
  3. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  4. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
  5. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,109
  6. 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,708
  7. 2025VGGT: Visual Geometry Grounded Transformercited by 1,539
  8. 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 927
  9. 2025Emerging Properties in Unified Multimodal Pretrainingcited by 897
  10. 2024Show-o: One Single Transformer to Unify Multimodal Understanding and Generationcited by 752
  11. 2024Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusioncited by 681
  12. 2024Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spacescited by 611
38 more
  1. 2023MotionCtrl: A Unified and Flexible Motion Controller for Video Generationcited by 580
  2. 2024Long Context Transfer from Language to Visioncited by 515
  3. 2024Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and Generationcited by 457
  4. 2019ScanRefer: 3D Object Localization in RGB-D Scans using Natural Languagecited by 452
  5. 2024ViewCrafter: Taming Video Diffusion Models for High-fidelity Novel View Synthesiscited by 403
  6. 2025BLIP3-o: A Family of Fully Open Unified Multimodal Models-Architecture, Training and Datasetcited by 376
  7. 2023An Embodied Generalist Agent in 3D Worldcited by 365
  8. 2021ScanQA: 3D Question Answering for Spatial Scene Understandingcited by 346
  9. 2022SQA3D: Situated Question Answering in 3D Scenescited by 314
  10. 2024LongVILA: Scaling Long-Context Visual Language Models for Long Videoscited by 309
  11. 2024VILA-U: a Unified Foundation Model Integrating Visual Understanding and Generationcited by 295
  12. 2025Show-o2: Improved Native Unified Multimodal Modelscited by 250
  13. 2023LucidDreamer: Domain-free Generation of 3D Gaussian Splatting Scenescited by 231
  14. 2025Spatial-MLLM: Boosting MLLM Capabilities in Visual-based Spatial Intelligencecited by 217
  15. 2024MetaMorph: Multimodal Understanding and Generation via Instruction Tuningcited by 206
  16. 2024WonderWorld: Interactive 3D Scene Generation from a Single Imagecited by 202
  17. 2024LLaVA-3D: A Simple yet Effective Pathway to Empowering LMMs with 3D-awarenesscited by 183
  18. 2025VLM-3R: Vision-Language Models Augmented with Instruction-Aligned 3D Reconstructioncited by 166
  19. 2023WonderJourney: Going from Anywhere to Everywherecited by 158
  20. 2024Video-3D LLM: Learning Position-Aware Video Representation for 3D Scene Understandingcited by 138
  21. 2025From Flatland to Space: Teaching Vision-Language Models to Perceive and Reason in 3Dcited by 138
  22. 2025Learning from Videos for 3D World: Enhancing MLLMs with 3D Vision Geometry Priorscited by 120
  23. 2024Grounded 3D-LLM with Referent Tokenscited by 115
  24. 2024SAT: Dynamic Spatial Aptitude Training for Multimodal Language Modelscited by 114
  25. 2025Voyager: Long-Range and World-Consistent Video Diffusion for Explorable 3D Scene Generationcited by 101
  26. 2025HunyuanWorld 1.0: Generating Immersive, Explorable, and Interactive 3D Worlds from Words or Pixelscited by 94
  27. 2024ChatScene: Knowledge-Enabled Safety-Critical Scenario Generation for Autonomous Vehiclescited by 93
  28. 2025Unified Multimodal Understanding and Generation Models: Advances, Challenges, and Opportunitiescited by 85
  29. 2025Harmonizing Visual Representations for Unified Multimodal Understanding and Generationcited by 74
  30. 2024You See it, You Got it: Learning 3D Creation on Pose-Free Videos at Scalecited by 65
  31. 2024Liquid: Language Models are Scalable and Unified Multi-modal Generatorscited by 63
  32. 2025Ross3D: Reconstructive Visual Instruction Tuning with 3D-Awarenesscited by 63
  33. 2025HERMES: A Unified Self-Driving World Model for Simultaneous 3D Scene Understanding and Generationcited by 56
  34. 2025FlexWorld: Progressively Expanding 3D Scenes for Flexiable-View Synthesiscited by 37
  35. 2025Universal Image Restoration Pre-training via Degradation Classificationcited by 35
  36. 2025StarGen: A Spatiotemporal Autoregression Framework with Video Diffusion Model for Scalable and Controllable Scene Generationcited by 26
  37. 2025Auto-Regressively Generating Multi-View Consistent Imagescited by 9
  38. 2025Enhancing Image Restoration Transformer via Adaptive Translation Equivariancecited by 5

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 17.

17 papers.

Sort:

By area:

  1. 2026MVGGT: Multimodal Visual Geometry Grounded Transformer for Multiview 3D Referring Expression Segmentationcited by 8
  2. 2026Tele-Omni: a Unified Multimodal Framework for Video Generation and Editingcited by 8
  3. 2026Geometry-as-context: Modulating Explicit 3D in Scene-consistent Video Generation to Geometry Contextcited by 8
  4. 2026Generation Models Know Space: Unleashing Implicit 3D Priors for Scene Understandingcited by 7
  5. 2026Bridging Information Asymmetry: A Hierarchical Framework for Deterministic Blind Face Restorationcited by 3
  6. 2026Towards Physics of Multimodal Pretraining: Knowledge Flow, Modality Synergy, Early Unification, and Recipescited by 3
  7. 2026What if? Emulative Simulation with World Models for Situated Reasoningcited by 2
  8. 2026Narrative Weaver: Towards Controllable Long-Range Visual Consistency with Multi-Modal Conditioningcited by 2
  9. 2026HYDRA: Representation Harmonized Tokenization for Multimodal Generation and Understandingcited by 2
  10. 2026Unpaired Image Deraining Using Reward-Guided Self-Reinforcement Strategycited by 2
  11. 2026FedUMM: A General Framework for Federated Learning with Unified Multimodal Modelscited by 0
  12. 2026Bridging Degradation Discrimination and Generation for Universal Image Restorationcited by 0
5 more
  1. 2026CoordRefer: Coordinate-Aware 3D Visual Grounding from Multiview Imagescited by 0
  2. 2026Fysiverse-3D-Vision Technical Report: Generating Executable 3D Worlds from Images through Unified Spatial Reasoningcited by 0
  3. 2026SpatialSpeak: QA-Native Reconstruction with Local and Global Context for Spatial Chain-of-Thought Reasoningcited by 0
  4. 2026Render to Reason: Novel-View Semantic Prediction Improves Spatial Understanding in VLMscited by 0
  5. 2026Can Learning to Answer Spatial Questions Improve Novel-View Generation?cited by 0