# PanoFuse: Panorama-Enhanced Vision-Language-Action Learning with Decoupled Semantic-Geometric Routing

arXiv: https://arxiv.org/abs/2609.31717 · HTML: https://mlsanity.com/paper/2609.31717/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in 3D and spatial-aware VLA, one of 13 directions in Vision-language-action models.
- Builds on 34 of its 42 references that are in the map.
- First posted on arXiv 21 Sep 2026.

## Placed in

- Vision-language-action models / [3D and spatial-aware VLA](https://mlsanity.com/vla/3d-and-spatial-aware-vla/)

## Builds on (most cited first, 30 of 34)

- [RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control](https://mlsanity.com/paper/2307.15818/) (2023)
- [OpenVLA: An Open-Source Vision-Language-Action Model](https://mlsanity.com/paper/2406.09246/) (2024)
- [π0: A Vision-Language-Action Flow Model for General Robot Control](https://mlsanity.com/paper/2410.24164/) (2024)
- [Octo: An Open-Source Generalist Robot Policy](https://mlsanity.com/paper/2405.12213/) (2024)
- [FAST: Efficient Action Tokenization for Vision-Language-Action Models](https://mlsanity.com/paper/2501.09747/) (2025)
- [CoT-VLA: Visual Chain-of-Thought Reasoning for Vision-Language-Action Models](https://mlsanity.com/paper/2503.22020/) (2025)
- [ForceVLA: Enhancing VLA Models with a Force-aware MoE for Contact-rich Manipulation](https://mlsanity.com/paper/2505.22159/) (2025)
- [4D-VLA: Spatiotemporal Vision-Language-Action Pretraining with Cross-Scene Calibration](https://mlsanity.com/paper/2506.22242/) (2025)
- [DepthVLA: Enhancing Vision-Language-Action Models with Depth-Aware Spatial Reasoning](https://mlsanity.com/paper/2510.13375/) (2025)
- [Review on Panoramic Imaging and Its Applications in Scene Understanding](https://mlsanity.com/paper/2205.05570/) (2022)
- [DeepPanoContext: Panoramic 3D Scene Understanding with Holistic Scene Context Graph and Relation-based Optimization](https://mlsanity.com/paper/2108.10743/) (2021)
- [Depth Any Panoramas: A Foundation Model for Panoramic Depth Estimation](https://mlsanity.com/paper/2512.16913/) (2025)
- [PanoSwin: a Pano-style Swin Transformer for Panorama Understanding](https://mlsanity.com/paper/2308.14726/) (2023)
- [StereoVLA: Enhancing Vision-Language-Action Models with Stereo Vision](https://mlsanity.com/paper/2512.21970/) (2025)
- [AsyncVLA: Asynchronous Flow Matching for Vision-Language-Action Models](https://mlsanity.com/paper/2511.14148/) (2025)
- [MonoDream: Monocular Vision-Language Navigation with Panoramic Dreaming](https://mlsanity.com/paper/2508.02549/) (2025)
- [VLA-Corrector: Lightweight Detect-and-Correct Inference for Adaptive Action Horizon](https://mlsanity.com/paper/2607.01804/) (2026)
- [BFA: Best-Feature-Aware Fusion for Multi-View Fine-grained Manipulation](https://mlsanity.com/paper/2502.11161/) (2025)
- [PanoNav: Mapless Zero-Shot Object Navigation with Panoramic Scene Parsing and Dynamic Memory](https://mlsanity.com/paper/2511.06840/) (2025)
- [Intercepting the Future: Latent-Space Predictive World Model for Dynamic VLA Manipulation](https://mlsanity.com/paper/2606.02486/) (2026)
- [Learning Panorama-Aware VLA for Mobile Manipulation with Whole-Body Teleoperation](https://mlsanity.com/paper/2608.02257/) (2026)
- [U-ARM : Ultra low-cost general teleoperation interface for robot manipulation](https://mlsanity.com/paper/2509.02437/) (2025)
- [VLA-LPAF: Lightweight Perspective-Adaptive Fusion for Vision-Language-Action to Enable More Unconstrained Robotic Manipulation](https://mlsanity.com/paper/2509.18183/) (2025)
- [TGM-VLA: Task-Guided Mixup for Sampling-Efficient and Robust Robotic Manipulation](https://mlsanity.com/paper/2603.00615/) (2026)
- [Navigation beyond Wayfinding: Robots Collaborating with Visually Impaired Users for Environmental Interactions](https://mlsanity.com/paper/2603.14216/) (2026)
- [Learning Action Manifold with Multi-view Latent Priors for Robotic Manipulation](https://mlsanity.com/paper/2605.11832/) (2026)
- [Guide, Think, Act: Interactive Embodied Reasoning in Vision-Language-Action Models](https://mlsanity.com/paper/2605.13632/) (2026)
- [LIBERO-Occ: Evaluating and Improving Vision-Language-Action Models under Scene-Induced Occlusion via Viewpoint Imagination](https://mlsanity.com/paper/2606.10862/) (2026)
- [People Tracking in Panoramic Video for Guiding Robots](https://mlsanity.com/paper/2206.02735/) (2022)
- [OmniDP: Beyond-FOV Large-Workspace Humanoid Manipulation with Omnidirectional 3D Perception](https://mlsanity.com/paper/2603.05355/) (2026)

## Abstract (arXiv)

Vision-Language-Action (VLA) policies have shown promising performance in language-conditioned robotic manipulation. However, most existing VLA systems rely on conventional perspective cameras with limited fields of view, often missing global scene context and leading to unreliable manipulation under visual occlusions, distractors, and unseen environments. In this work, we propose PanoFuse, a panorama-enhanced VLA framework that complements local manipulation observations with global panoramic perception. PanoFuse introduces a dedicated panoramic branch that leverages a pretrained panoramic foundation model to extract complementary semantic and geometric representations from omnidirectional observations. Rather than directly mixing these heterogeneous features, we introduce Decoupled Semantic-Geometric Routing (DSGR), which maintains semantic and geometric representations as separate context streams and selectively routes both to downstream state and action representations through structured block-wise attention. This design provides the action expert with global spatial context while preserving task-relevant semantic information from the pretrained VLA backbone. We further develop a synchronized data collection pipeline and construct a new real-world manipulation dataset containing panoramic RGB observations, wrist-view images, language instructions, robot states, and actions. Across seven evaluation settings, PanoFuse achieves an average success rate of 52.9%, outperforming the evaluated baselines and achieving consistent gains under novel-object, unseen-background, and distractor-rich settings. Code and data will be released publicly at https://xux-hnu.github.io/PanoFuse.