mlsanityData to 14 Oct 2026

PanoWorld: Towards Spatial Supersensing in 360^circ Panorama World

arXiv 2605.13169 ↗First posted 13 May 2026cs.CV cs.AIcited by 5

Not on the map. Why?

Abstract (arXiv)

Multimodal large laboratory models (MLLMs) still struggle with spatial understanding under the dominant perspective-image paradigm, which inherits the narrow field of view of human-like perception. For navigation, robotic search, and 3D scene understanding, 360-degree panoramic sensing offers a form of supersensing by capturing the entire surrounding environment at once. However, existing MLLM pipelines typically decompose panoramas into multiple perspective views, leaving the spherical structure of equirectangular projection (ERP) largely implicit. In this paper, we study pano-native understanding, which requires an MLLM to reason over an ERP panorama as a continuous, observer-centered space. To this end, we first define the key abilities for pano-native understanding, including semantic anchoring, spherical localization, reference-frame transformation, and depth-aware 3D spatial reasoning. We then build a large-scale metadata construction pipeline that converts mixed-source ERP panoramas into geometry-aware, language-grounded, and depth-aware supervision, and instantiate these signals as capability-aligned instruction tuning data. On the model side, we introduce PanoWorld with Spherical Spatial Cross-Attention, which injects spherical geometry into the visual stream. We further construct PanoSpace-Bench, a diagnostic benchmark for evaluating ERP-native spatial reasoning. Experiments show that PanoWorld substantially outperforms both proprietary and open-source baselines on PanoSpace-Bench, H* Bench, and R2R-CE Val-Unseen benchmarks. These results demonstrate that robust panoramic reasoning requires dedicated pano-native supervision and geometry-aware model adaptation. All source code and proposed data will be publicly released.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 50 of its 59 references that are in the map. First posted on arXiv 13 May 2026.

Builds on

50 of 59 references.

References · 50

  1. 2024GPT-4o System Cardcited by 4,588
  2. 2024Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMscited by 838
  3. 2024SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilitiescited by 706
  4. 2024Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spacescited by 611
  5. 2020Beyond the Nav-Graph: Vision-and-Language Navigation in Continuous Environmentscited by 441
  6. 2023An Embodied Generalist Agent in 3D Worldcited by 365
  7. 2025VL-Rethinker: Incentivizing Self-Reflection of Vision-Language Models with Reinforcement Learningcited by 323
  8. 2022SQA3D: Situated Question Answering in 3D Scenescited by 314
  9. 2024SpatialRGPT: Grounded Spatial Reasoning in Vision Language Modelscited by 311
  10. 2019Structured3D: A Large Photo-realistic Dataset for Structured 3D Modelingcited by 286
  11. 2025Pixel Reasoner: Incentivizing Pixel-Space Reasoning with Curiosity-Driven Reinforcement Learningcited by 278
  12. 2024LLaVA-3D: A Simple yet Effective Pathway to Empowering LMMs with 3D-awarenesscited by 183
38 more
  1. 2024Video-3D LLM: Learning Position-Aware Video Representation for 3D Scene Understandingcited by 138
  2. 2025Cambrian-S: Towards Spatial Supersensing in Videocited by 134
  3. 2021Waypoint Models for Instruction-guided Navigation in Continuous Environmentscited by 129
  4. 2023GridMM: Grid Memory Map for Vision-and-Language Navigationcited by 124
  5. 2025StreamVLN: Streaming Vision-and-Language Navigation via SlowFast Context Modelingcited by 118
  6. 2022PanoFormer: Panorama Transformer for Indoor 360 Depth Estimationcited by 95
  7. 2021Pano-AVQA: Grounded Audio-Visual Question Answering on 360^circ Videoscited by 86
  8. 2022Sim-2-Sim Transfer for Vision-and-Language Navigation in Continuous Environmentscited by 86
  9. 2023DREAMWALKER: Mental Planning for Continuous Vision-Language Navigationcited by 85
  10. 2025How to Enable LLM with 3D Capacity? A Survey of Spatial Reasoning in LLMcited by 42
  11. 2024Depth Anywhere: Enhancing 360 Monocular Depth Estimation via Perspective Distillation and Unlabeled Data Augmentationcited by 38
  12. 2024360+x: A Panoptic Multi-modal Scene Understanding Datasetcited by 35
  13. 2025Multimodal Spatial Reasoning in the Large Model Era: A Survey and Benchmarkscited by 33
  14. 2025OmniSAM: Omnidirectional Segment Anything Model for UDA in Panoramic Semantic Segmentationcited by 28
  15. 2023OSRT: Omnidirectional Image Super-Resolution with Distortion-aware Transformercited by 26
  16. 2025One Flight Over the Gap: A Survey from Perspective to Panoramic Visioncited by 25
  17. 2025DA2: Depth Anything in Any Directioncited by 25
  18. 2025Depth Any Panoramas: A Foundation Model for Panoramic Depth Estimationcited by 25
  19. 2025DiT360: High-Fidelity Panoramic Image Generation via Hybrid Trainingcited by 24
  20. 2025Dynam3D: Dynamic Layered 3D Tokens Empower VLM for Vision-and-Language Navigationcited by 23
  21. 2025Are Multimodal Large Language Models Ready for Omnidirectional Spatial Reasoning?cited by 23
  22. 2025Spatial Reasoning in Multimodal Large Language Models: A Survey of Tasks, Benchmarks and Methodscited by 23
  23. 2024PanDA: Towards Panoramic Depth Anything with Unlabeled Panoramas and Mobius Spatial Augmentationcited by 22
  24. 2023PanoSwin: a Pano-style Swin Transformer for Panorama Understandingcited by 20
  25. 2026OmniRoam: World Wandering via Long-Horizon Panoramic Video Generationcited by 16
  26. 2025Thinking in 360deg: Humanoid Visual Search in the Wildcited by 15
  27. 2025WeDetect: Fast Open-Vocabulary Object Detection as Retrievalcited by 14
  28. 2026RationalRewards: Reasoning Rewards Scale Visual Generation Both Training and Test Timecited by 14
  29. 2025Towards Omnidirectional Reasoning with 360-R1: A Dataset, Benchmark, and GRPO-based Methodcited by 13
  30. 2026PanoVGGT: Feed-Forward 3D Reconstruction from Panoramic Imagerycited by 13
  31. 2025ODI-Bench: Can MLLMs Understand Immersive Omnidirectional Environments?cited by 12
  32. 2025AirSim360: A Panoramic Simulation Platform within Drone Viewcited by 12
  33. 2025Dense360: Dense Understanding from Omnidirectional Panoramascited by 9
  34. 2025From Illusion to Intention: Visual Rationale Learning for Vision-Language Reasoningcited by 9
  35. 2026PanoEnv: Exploring 3D Spatial Intelligence in Panoramic Environments with Reinforcement Learningcited by 7
  36. 2025Omnidirectional Spatial Modeling from Correlated Panoramascited by 5
  37. 2026textscNaVIDA: Vision-Language Navigation with Inverse Dynamics Augmentationcited by 5
  38. 2026More than the Sum: Panorama-Language Models for Adverse Omni-Scenescited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 5.

5 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026EAGLE-360: Embodied Active Global-to-Local Exploration in 360^circ
  2. 2026Mover360: Controllable Object Manipulation in 360deg Panoramic Images
  3. 2026Pano2World: End-to-End 3D Generation via Unified Multi-View Sequences
  4. 2026OneCanvas: 3D Scene Understanding via Panoramic Reprojection
  5. 2026OmniAct3D: Leveraging Foundation Geometry and Evidence-Grounded Reasoning for Panoramic 3D Detection
  6. 2026SPW-Nav: A Streaming Panoramic World Model for Language-Guided Navigation
  7. 2025PanoGrounder: Bridging 2D and 3D with Panoramic Scene Representations for VLM-based 3D Visual Grounding
  8. 2026Bridging Language and Spherical Space: Object-Centric Control for Text-to-Panorama Generation