# PanoSeg3R: Feed-Forward 3D Semantic Segmentation for Panoramic Images with an Automatic Data Curation Pipeline

arXiv: https://arxiv.org/abs/2609.22687 · HTML: https://mlsanity.com/paper/2609.22687/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 44 references that are in the map.
- First posted on arXiv 19 Sep 2026.

## Builds on (most cited first, 30 of 30)

- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [Qwen3-VL Technical Report](https://mlsanity.com/paper/2511.21631/) (2025)
- [Masked-attention Mask Transformer for Universal Image Segmentation](https://mlsanity.com/paper/2112.01527/) (2021)
- [Decoupled Weight Decay Regularization](https://mlsanity.com/paper/1711.05101/) (2017)
- [Vision Transformers for Dense Prediction](https://mlsanity.com/paper/2103.13413/) (2021)
- [SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Features](https://mlsanity.com/paper/2502.14786/) (2025)
- [SAM 3: Segment Anything with Concepts](https://mlsanity.com/paper/2511.16719/) (2025)
- [ScanNet++: A High-Fidelity Dataset of 3D Indoor Scenes](https://mlsanity.com/paper/2308.11417/) (2023)
- [CMX: Cross-Modal Fusion for RGB-X Semantic Segmentation with Transformers](https://mlsanity.com/paper/2203.04838/) (2022)
- [MUSt3R: Multi-view Network for Stereo 3D Reconstruction](https://mlsanity.com/paper/2503.01661/) (2025)
- [UniFuse: Unidirectional Fusion for 360circ Panorama Depth Estimation](https://mlsanity.com/paper/2102.03550/) (2021)
- [Spherical CNNs on Unstructured Grids](https://mlsanity.com/paper/1901.02039/) (2019)
- [Tangent Images for Mitigating Spherical Distortion](https://mlsanity.com/paper/1912.09390/) (2019)
- [Large Spatial Model: End-to-end Unposed Images to Semantic 3D](https://mlsanity.com/paper/2410.18956/) (2024)
- [UniK3D: Universal Camera Monocular 3D Estimation](https://mlsanity.com/paper/2503.16591/) (2025)
- [Orientation-aware Semantic Segmentation on Icosahedron Spheres](https://mlsanity.com/paper/1907.12849/) (2019)
- [Behind Every Domain There is a Shift: Adapting Distortion-aware Vision Transformers for Panoramic Semantic Segmentation](https://mlsanity.com/paper/2207.11860/) (2022)
- [SpherePHD: Applying CNNs on a Spherical PolyHeDron Representation of 360 degree Images](https://mlsanity.com/paper/1811.08196/) (2018)
- [IGGT: Instance-Grounded Geometry Transformer for Semantic 3D Reconstruction](https://mlsanity.com/paper/2510.22706/) (2025)
- [SGAT4PASS: Spherical Geometry-Aware Transformer for PAnoramic Semantic Segmentation](https://mlsanity.com/paper/2306.03403/) (2023)
- [PanSt3R: Multi-view Consistent Panoptic Segmentation](https://mlsanity.com/paper/2506.21348/) (2025)
- [Single Frame Semantic Segmentation Using Multi-Modal Spherical Images](https://mlsanity.com/paper/2308.09369/) (2023)
- [SIU3R: Simultaneous Scene Understanding and 3D Reconstruction Beyond Feature Alignment](https://mlsanity.com/paper/2507.02705/) (2025)
- [PICCOLO: Point Cloud-Centric Omnidirectional Localization](https://mlsanity.com/paper/2108.06545/) (2021)
- [BridgeNet: Comprehensive and Effective Feature Interactions via Bridge Feature for Multi-task Dense Predictions](https://mlsanity.com/paper/2312.13514/) (2023)
- [MTPano: Multi-Task Panoramic Scene Understanding via Label-Free Integration of Dense Prediction Priors](https://mlsanity.com/paper/2602.05330/) (2026)
- [PanoSAMic: Panoramic Image Segmentation from SAM Feature Encoding and Dual View Fusion](https://mlsanity.com/paper/2601.07447/) (2026)
- [Wid3R: Wide Field-of-View 3D Reconstruction via Camera Model Conditioning](https://mlsanity.com/paper/2602.05321/) (2026)
- [IM360: Large-scale Indoor Mapping with 360 Cameras](https://mlsanity.com/paper/2502.12545/) (2025)

## Built on by (most cited first, 1 of 1)

- [S4VY: Segment Anything in Feed-Forward 4D Visual Geometry](https://mlsanity.com/paper/2609.36875/) (2026)

## Abstract (arXiv)

We present PanoSeg3R, a feed-forward framework for 3D panoramic semantic segmentation. Unlike existing methods designed for perspective inputs, PanoSeg3R jointly predicts 3D geometry and multi-view semantic segmentation in one single forward pass. Built upon a pretrained reconstruction backbone that supports panoramic images, our approach extends feed-forward 3D reconstruction with a query-based mask decoder. Furthermore, we introduce an automatic panorama data curation pipeline that leverages the complementary strengths of off-the-shelf foundation models to generate reliable pseudo semantic annotations, substantially expanding the training data and improving zero-shot generalization. PanoSeg3R achieves state-of-the-art performance on panoramic 3D semantic segmentation, improving 3D mIoU by up to 16.02 on ScanNet++, while the curated training data further improves zero-shot performance by up to 4.26 and 43.28 mIoU on Stanford2D3D and ToF-360, respectively. Website: https://harryyoon777.github.io/PanoSeg3R/