mlsanityData to 7 Oct 2026

SphereUFormer: A U-Shaped Transformer for Spherical 360 Perception

arXiv 2412.06968 ↗First posted 9 Dec 2024cs.CVcited by 12

Not on the map. Why?

Abstract (arXiv)

This paper proposes a novel method for omnidirectional 360degree perception. Most common previous methods relied on equirectangular projection. This representation is easily applicable to 2D operation layers but introduces distortions into the image. Other methods attempted to remove the distortions by maintaining a sphere representation but relied on complicated convolution kernels that failed to show competitive results. In this work, we introduce a transformer-based architecture that, by incorporating a novel ``Spherical Local Self-Attention'' and other spherically-oriented modules, successfully operates in the spherical domain and outperforms the state-of-the-art in 360degree perception benchmarks for depth estimation and semantic segmentation.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 36 of its 56 references that are in the map. First posted on arXiv 9 Dec 2024.

Builds on

36 of 56 references.

References · 36

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2017Graph Attention Networkscited by 8,854
  4. 2020End-to-End Object Detection with Transformerscited by 7,384
  5. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
  6. 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,923
  7. 2021Vision Transformers for Dense Predictioncited by 1,789
  8. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,354
  9. 2021Per-Pixel Classification is Not All You Need for Semantic Segmentationcited by 1,148
  10. 2015Effective Approaches to Attention-based Neural Machine Translationcited by 1,024
  11. 2021Uformer: A General U-Shaped Transformer for Image Restorationcited by 864
  12. 2020Geom-GCN: Geometric Graph Convolutional Networkscited by 821
24 more
  1. 2017Joint 2D-3D-Semantic Data for Indoor Scene Understandingcited by 404
  2. 2019Structured3D: A Large Photo-realistic Dataset for Structured 3D Modelingcited by 285
  3. 2020HoHoNet: 360 Indoor Holistic Understanding with Latent Horizontal Featurescited by 104
  4. 2018OmniDepth: Dense Depth Estimation for Indoors Spherical Panoramascited by 98
  5. 2022PanoFormer: Panorama Transformer for Indoor 360 Depth Estimationcited by 94
  6. 2021UniFuse: Unidirectional Fusion for 360circ Panorama Depth Estimationcited by 91
  7. 2019Tangent Images for Mitigating Spherical Distortioncited by 79
  8. 2019Spherical CNNs on Unstructured Gridscited by 76
  9. 2018Cube Padding for Weakly-Supervised Saliency Prediction in 360deg Videoscited by 66
  10. 2020DeepSphere: a graph-based spherical CNNcited by 65
  11. 2021Subdivision-Based Mesh Convolution Networkscited by 60
  12. 2021360MonoDepth: High-Resolution 360deg Monocular Depth Estimationcited by 60
  13. 2022OmniFusion: 360 Monocular Depth Estimation via Geometry-Aware Fusioncited by 60
  14. 2019Orientation-aware Semantic Segmentation on Icosahedron Spherescited by 52
  15. 2023EGformer: Equirectangular Geometry-biased Transformer for 360 Depth Estimationcited by 42
  16. 2018SpherePHD: Applying CNNs on a Spherical PolyHeDron Representation of 360 degree Imagescited by 40
  17. 2021Pano3D: A Holistic Benchmark and a Solid Baseline for 360o Depth Estimationcited by 37
  18. 2023HRDFuse: Monocular 360degDepth Estimation by Collaboratively Learning Holistic-with-Regional Depth Distributionscited by 37
  19. 2022BiFuse++: Self-supervised and Efficient Bi-projection Fusion for 360 Depth Estimationcited by 35
  20. 2024Elite360D: Towards Efficient 360 Depth Estimation via Semantic- and Distance-Aware Bi-Projection Fusioncited by 31
  21. 2021Improving 360 Monocular Depth Estimation via Non-local Dense Prediction Transformer and Joint Supervised and Self-supervised Learningcited by 19
  22. 2023HEAL-SWIN: A Vision Transformer On The Spherecited by 18
  23. 2023Single Frame Semantic Segmentation Using Multi-Modal Spherical Imagescited by 18
  24. 2022SphereDepth: Panorama Depth Estimation from Spherical Domaincited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 3; 2026: 9.

12 papers.

  1. 2025Depth Any Panoramas: A Foundation Model for Panoramic Depth Estimationcited by 24
  2. 2025Attention on the Spherecited by 9
  3. 2026EAGOR: Embodied Reasoning in Omni-directioncited by 3
  4. 2025OmniTrack++: Omnidirectional Multi-Object Tracking by Learning Large-FoV Trajectory Feedbackcited by 2
  5. 2026MARA: Continuous SE(3)-Equivariant Attention for Molecular Force Fieldscited by 1
  6. 2026SO3UFormer: Learning Intrinsic Spherical Features for Rotation-Robust Panoramic Dense Predictioncited by 1
  7. 2026EAGLE-360: Embodied Active Global-to-Local Exploration in 360^circcited by 1
  8. 2026AdapToPASS: Ambiguity-aware Adaptive Spherical Transformer for Panoramic Semantic Segmentationcited by 1
  9. 2026Neural Operators for Biomedical Spherical Heterogeneitycited by 0
  10. 2026Panoramic Scene Understanding: A Survey from Distortion-Aware Engineering to Sphere-Native Modelingcited by 0
  11. 2026SphMind: Towards Robust, Training-Free VLM-based Spatial Reasoning with a 360 Cameracited by 0
  12. 2026SCCM: Spherically Consistent Coarse Matching for ERP Dense Feature Correspondencecited by 0