# GoodSAM: Bridging Domain and Capacity Gaps via Segment Anything Model for Distortion-aware Panoramic Semantic Segmentation

arXiv: https://arxiv.org/abs/2403.16370 · HTML: https://mlsanity.com/paper/2403.16370/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 46 references that are in the map.
- First posted on arXiv 25 Mar 2024.

## Builds on (most cited first, 29 of 29)

- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers](https://mlsanity.com/paper/2105.15203/) (2021)
- [FLAVA: A Foundational Language And Vision Alignment Model](https://mlsanity.com/paper/2112.04482/) (2021)
- [Medical SAM Adapter: Adapting Segment Anything Model for Medical Image Segmentation](https://mlsanity.com/paper/2304.12620/) (2023)
- [Track Anything: Segment Anything Meets Videos](https://mlsanity.com/paper/2304.11968/) (2023)
- [Inpaint Anything: Segment Anything Meets Image Inpainting](https://mlsanity.com/paper/2304.06790/) (2023)
- [Segment and Track Anything](https://mlsanity.com/paper/2305.06558/) (2023)
- [Curriculum Domain Adaptation for Semantic Segmentation of Urban Scenes](https://mlsanity.com/paper/1707.09465/) (2017)
- [Knowledge Adaptation for Efficient Semantic Segmentation](https://mlsanity.com/paper/1903.04688/) (2019)
- [Prototypical Cross-domain Self-supervised Learning for Few-shot Unsupervised Domain Adaptation](https://mlsanity.com/paper/2103.16765/) (2021)
- [Bending Reality: Distortion-aware Transformers for Adapting to Panoramic Semantic Segmentation](https://mlsanity.com/paper/2203.01452/) (2022)
- [A Comprehensive Survey on Segment Anything Model for Vision and Beyond](https://mlsanity.com/paper/2305.08196/) (2023)
- [LVM-Med: Learning Large-Scale Self-Supervised Vision Models for Medical Imaging via Second-order Graph Matching](https://mlsanity.com/paper/2306.11925/) (2023)
- [Patch-Mix Transformer for Unsupervised Domain Adaptation: A Game Perspective](https://mlsanity.com/paper/2303.13434/) (2023)
- [Segment Anything Model (SAM) Enhanced Pseudo Labels for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2305.05803/) (2023)
- [Both Style and Distortion Matter: Dual-Path Unsupervised Domain Adaptation for Panoramic Semantic Segmentation](https://mlsanity.com/paper/2303.14360/) (2023)
- [Behind Every Domain There is a Shift: Adapting Distortion-aware Vision Transformers for Panoramic Semantic Segmentation](https://mlsanity.com/paper/2207.11860/) (2022)
- [Deep Learning for Omnidirectional Vision: A Survey and New Perspectives](https://mlsanity.com/paper/2205.10468/) (2022)
- [Look at the Neighbor: Distortion-aware Unsupervised Domain Adaptation for Panoramic Semantic Segmentation](https://mlsanity.com/paper/2308.05493/) (2023)
- [DensePASS: Dense Panoramic Semantic Segmentation via Unsupervised Domain Adaptation with Attention-Augmented Context Exchange](https://mlsanity.com/paper/2108.06383/) (2021)
- [Transfer beyond the Field of View: Dense Panoramic Semantic Segmentation via Unsupervised Domain Adaptation](https://mlsanity.com/paper/2110.11062/) (2021)
- [Semantics, Distortion, and Style Matter: Towards Source-free UDA for Panoramic Segmentation](https://mlsanity.com/paper/2403.12505/) (2024)
- [DeepPanoContext: Panoramic 3D Scene Understanding with Holistic Scene Context Graph and Relation-based Optimization](https://mlsanity.com/paper/2108.10743/) (2021)
- [A Good Student is Cooperative and Reliable: CNN-Transformer Collaborative Learning for Semantic Segmentation](https://mlsanity.com/paper/2307.12574/) (2023)
- [DS-PASS: Detail-Sensitive Panoramic Annular Semantic Segmentation through SwaftNet for Surrounding Sensing](https://mlsanity.com/paper/1909.07721/) (2019)
- [SGAT4PASS: Spherical Geometry-Aware Transformer for PAnoramic Semantic Segmentation](https://mlsanity.com/paper/2306.03403/) (2023)
- [Knowledge Adaptation: Teaching to Adapt](https://mlsanity.com/paper/1702.02052/) (2017)
- [Semantic Segmentation of Panoramic Images Using a Synthetic Dataset](https://mlsanity.com/paper/1909.00532/) (2019)

## Built on by (most cited first, 28 of 28)

- [360SFUDA++: Towards Source-free UDA for Panoramic Segmentation by Learning Reliable Category Prototypes](https://mlsanity.com/paper/2404.16501/) (2024)
- [OmniSAM: Omnidirectional Segment Anything Model for UDA in Panoramic Semantic Segmentation](https://mlsanity.com/paper/2503.07098/) (2025)
- [A Survey of Representation Learning, Optimization Strategies, and Applications for Omnidirectional Vision](https://mlsanity.com/paper/2502.10444/) (2025)
- [Are Multimodal Large Language Models Ready for Omnidirectional Spatial Reasoning?](https://mlsanity.com/paper/2505.11907/) (2025)
- [Segment Anything with Multiple Modalities](https://mlsanity.com/paper/2408.09085/) (2024)
- [PANORAMA: The Rise of Omnidirectional Vision in the Embodied AI Era](https://mlsanity.com/paper/2509.12989/) (2025)
- [GoodSAM++: Bridging Domain and Capacity Gaps via Segment Anything Model for Panoramic Semantic Segmentation](https://mlsanity.com/paper/2408.09115/) (2024)
- [Reducing Unimodal Bias in Multi-Modal Semantic Segmentation with Multi-Scale Functional Entropy Regularization](https://mlsanity.com/paper/2505.06635/) (2025)
- [Multi-source Domain Adaptation for Panoramic Semantic Segmentation](https://mlsanity.com/paper/2408.16469/) (2024)
- [Leader360V: The Large-scale, Real-world 360 Video Dataset for Multi-task Learning in Diverse Environment](https://mlsanity.com/paper/2506.14271/) (2025)
- [PanoEnv: Exploring 3D Spatial Intelligence in Panoramic Environments with Reinforcement Learning](https://mlsanity.com/paper/2602.21992/) (2026)
- [Panoramic Multimodal Semantic Occupancy Prediction for Quadruped Robots](https://mlsanity.com/paper/2603.13108/) (2026)
- [Unlocking Constraints: Source-Free Occlusion-Aware Seamless Segmentation](https://mlsanity.com/paper/2506.21198/) (2025)
- [OmniTrack++: Omnidirectional Multi-Object Tracking by Learning Large-FoV Trajectory Feedback](https://mlsanity.com/paper/2511.00510/) (2025)
- [E-SAM: Training-Free Segment Every Entity Model](https://mlsanity.com/paper/2503.12094/) (2025)
- [Panoramic Out-of-Distribution Segmentation](https://mlsanity.com/paper/2505.03539/) (2025)
- [Rein++: Efficient Generalization and Adaptation for Semantic Segmentation with Vision Foundation Models](https://mlsanity.com/paper/2508.01667/) (2025)
- [O3N: Omnidirectional Open-Vocabulary Occupancy Prediction for Embodied Intelligent Robotics](https://mlsanity.com/paper/2603.12144/) (2026)
- [Seeing Beyond: Extrapolative Domain Adaptive Panoramic Segmentation](https://mlsanity.com/paper/2603.15475/) (2026)
- [PanoSAM2: Lightweight Distortion- and Memory-aware Adaptions of SAM2 for 360 Video Object Segmentation](https://mlsanity.com/paper/2604.07901/) (2026)
- [Panoramic Scene Understanding: A Survey from Distortion-Aware Engineering to Sphere-Native Modeling](https://mlsanity.com/paper/2606.27745/) (2026)
- [AdapToPASS: Ambiguity-aware Adaptive Spherical Transformer for Panoramic Semantic Segmentation](https://mlsanity.com/paper/2608.29081/) (2026)
- [Rethinking Knowledge in Distillation: An In-context Sample Retrieval Perspective](https://mlsanity.com/paper/2501.07040/) (2025)
- [Partial CLIP is Enough: Chimera-Seg for Zero-shot Semantic Segmentation](https://mlsanity.com/paper/2506.22032/) (2025)
- [SAP: Segment Any 4K Panorama](https://mlsanity.com/paper/2603.12759/) (2026)
- [FullCircle: Effortless 3D Reconstruction from Casual 360^circ Captures](https://mlsanity.com/paper/2603.22572/) (2026)
- [Instance-Aware Knowledge Distillation for Semi-Supervised Learning of an On-Board Multi-Task Dense Prediction Model for Collision Avoidance System](https://mlsanity.com/paper/2606.16414/) (2026)
- [EmbPASS: Towards Cross-Embodiment Open Panoramic Segmentation](https://mlsanity.com/paper/2610.03248/) (2026)

## Abstract (arXiv)

This paper tackles a novel yet challenging problem: how to transfer knowledge from the emerging Segment Anything Model (SAM) -- which reveals impressive zero-shot instance segmentation capacity -- to learn a compact panoramic semantic segmentation model, i.e., student, without requiring any labeled data. This poses considerable challenges due to SAM's inability to provide semantic labels and the large capacity gap between SAM and the student. To this end, we propose a novel framework, called GoodSAM, that introduces a teacher assistant (TA) to provide semantic information, integrated with SAM to generate ensemble logits to achieve knowledge transfer. Specifically, we propose a Distortion-Aware Rectification (DAR) module that first addresses the distortion problem of panoramic images by imposing prediction-level consistency and boundary enhancement. This subtly enhances TA's prediction capacity on panoramic images. DAR then incorporates a cross-task complementary fusion block to adaptively merge the predictions of SAM and TA to obtain more reliable ensemble logits. Moreover, we introduce a Multi-level Knowledge Adaptation (MKA) module to efficiently transfer the multi-level feature knowledge from TA and ensemble logits to learn a compact student model. Extensive experiments on two benchmarks show that our GoodSAM achieves a remarkable +3.75% mIoU improvement over the state-of-the-art (SOTA) domain adaptation methods. Also, our most lightweight model achieves comparable performance to the SOTA methods with only 3.7M parameters.