# AdapToPASS: Ambiguity-aware Adaptive Spherical Transformer for Panoramic Semantic Segmentation

arXiv: https://arxiv.org/abs/2608.29081 · HTML: https://mlsanity.com/paper/2608.29081/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 40 of its 63 references that are in the map.
- First posted on arXiv 29 Aug 2026.

## Builds on (most cited first, 30 of 40)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers](https://mlsanity.com/paper/2105.15203/) (2021)
- [SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation](https://mlsanity.com/paper/1511.00561/) (2015)
- [Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers](https://mlsanity.com/paper/2012.15840/) (2020)
- [Class-Balanced Loss Based on Effective Number of Samples](https://mlsanity.com/paper/1901.05555/) (2019)
- [Spherical CNNs](https://mlsanity.com/paper/1801.10130/) (2018)
- [Joint 2D-3D-Semantic Data for Indoor Scene Understanding](https://mlsanity.com/paper/1702.01105/) (2017)
- [Gauge Equivariant Convolutional Networks and the Icosahedral CNN](https://mlsanity.com/paper/1902.04615/) (2019)
- [Learning SO(3) Equivariant Representations with Spherical CNNs](https://mlsanity.com/paper/1711.06721/) (2017)
- [PanoFormer: Panorama Transformer for Indoor 360 Depth Estimation](https://mlsanity.com/paper/2203.09283/) (2022)
- [Bending Reality: Distortion-aware Transformers for Adapting to Panoramic Semantic Segmentation](https://mlsanity.com/paper/2203.01452/) (2022)
- [Orientation-aware Semantic Segmentation on Icosahedron Spheres](https://mlsanity.com/paper/1907.12849/) (2019)
- [Capturing Omni-Range Context for Omnidirectional Segmentation](https://mlsanity.com/paper/2103.05687/) (2021)
- [Behind Every Domain There is a Shift: Adapting Distortion-aware Vision Transformers for Panoramic Semantic Segmentation](https://mlsanity.com/paper/2207.11860/) (2022)
- [Active Boundary Loss for Semantic Segmentation](https://mlsanity.com/paper/2102.02696/) (2021)
- [Deep Learning for Omnidirectional Vision: A Survey and New Perspectives](https://mlsanity.com/paper/2205.10468/) (2022)
- [Elite360D: Towards Efficient 360 Depth Estimation via Semantic- and Distance-Aware Bi-Projection Fusion](https://mlsanity.com/paper/2403.16376/) (2024)
- [GoodSAM: Bridging Domain and Capacity Gaps via Segment Anything Model for Distortion-aware Panoramic Semantic Segmentation](https://mlsanity.com/paper/2403.16370/) (2024)
- [SGAT4PASS: Spherical Geometry-Aware Transformer for PAnoramic Semantic Segmentation](https://mlsanity.com/paper/2306.03403/) (2023)
- [A Survey of Representation Learning, Optimization Strategies, and Applications for Omnidirectional Vision](https://mlsanity.com/paper/2502.10444/) (2025)
- [PanDA: Towards Panoramic Depth Anything with Unlabeled Panoramas and Mobius Spatial Augmentation](https://mlsanity.com/paper/2406.13378/) (2024)
- [Learning from Noisy Labels with Decoupled Meta Label Purifier](https://mlsanity.com/paper/2302.06810/) (2023)
- [HEAL-SWIN: A Vision Transformer On The Sphere](https://mlsanity.com/paper/2307.07313/) (2023)
- [Single Frame Semantic Segmentation Using Multi-Modal Spherical Images](https://mlsanity.com/paper/2308.09369/) (2023)
- [M"obius Convolutions for Spherical CNNs](https://mlsanity.com/paper/2201.12212/) (2022)
- [SphereUFormer: A U-Shaped Transformer for Spherical 360 Perception](https://mlsanity.com/paper/2412.06968/) (2024)
- [Data augmentation with Mobius transformations](https://mlsanity.com/paper/2002.02917/) (2020)
- [Leader360V: The Large-scale, Real-world 360 Video Dataset for Multi-task Learning in Diverse Environment](https://mlsanity.com/paper/2506.14271/) (2025)

## Built on by (most cited first, 1 of 1)

- [SphMind: Towards Robust, Training-Free VLM-based Spatial Reasoning with a 360 Camera](https://mlsanity.com/paper/2609.33462/) (2026)

## Abstract (arXiv)

Spherical Transformers have emerged as a promising framework for panoramic semantic segmentation (PASS) by operating directly on spherical geometry and alleviating projection-induced distortions. However, existing architectures often assume canonical spherical structure and stable viewpoints, which are frequently violated in real-world imagery due to unconstrained camera motion, introducing contextual and geometric ambiguity. Consequently, they lack adaptive mechanisms to handle such ambiguity, limiting robustness to unseen spherical transformations. In contrast, biological perception is inherently ambiguity-aware, adapting to fluctuations in cue reliability caused by geometric and contextual variations to maintain stable interpretation under complex transformations. Motivated by this, we first systematically analyze existing PASS architectures under various unseen spherical transformations. We then introduce AdapToPASS, a novel bio-inspired Spherical Transformer that adaptively models contextual and geometric ambiguities for robust PASS. At its core, Adaptive Spherical Attention (AdaSpA) blocks dynamically modulate attention according to local contextual ambiguity, mimicking adaptive, context-driven biological perception. To address geometric ambiguity, AdapToPASS employs Bifocal Spherical Representation to balance field of view and spatial resolution, together with boundary supervision inspired by the boundary-sensitive nature of biological vision. Across indoor and outdoor semantic segmentation, AdapToPASS consistently outperforms prior state-of-the-art methods. Under unseen spherical transformations, it surpasses the next-best method by +13.38% relative mIoU on Stanford2D3D and +18.77% on WildPASS. We further introduce AdapToPASS-Swift, a lightweight variant with fewer than 2M parameters, which surpasses compact baselines while retaining robustness to spherical transformations.