mlsanityData to 7 Oct 2026

360BEV: Panoramic Semantic Mapping for Indoor Bird's-Eye View

arXiv 2303.11910 ↗First posted 21 Mar 2023cs.CVcited by 18

Not on the map. Why?

Abstract (arXiv)

Seeing only a tiny part of the whole is not knowing the full circumstance. Bird's-eye-view (BEV) perception, a process of obtaining allocentric maps from egocentric views, is restricted when using a narrow Field of View (FoV) alone. In this work, mapping from 360deg panoramas to BEV semantics, the 360BEV task, is established for the first time to achieve holistic representations of indoor scenes in a top-down view. Instead of relying on narrow-FoV image sequences, a panoramic image with depth information is sufficient to generate a holistic BEV semantic map. To benchmark 360BEV, we present two indoor datasets, 360BEV-Matterport and 360BEV-Stanford, both of which include egocentric panoramic images and semantic segmentation labels, as well as allocentric semantic maps. Besides delving deep into different mapping paradigms, we propose a dedicated solution for panoramic semantic mapping, namely 360Mapper. Through extensive experiments, our methods achieve 44.32% and 45.78% in mIoU on both datasets respectively, surpassing previous counterparts with gains of +7.60% and +9.70% in mIoU. Code and datasets are available at the project page: https://jamycheung.github.io/360BEV.html.

Key facts. Cited by 18 papers in the map (data through 7 Oct 2026). Builds on 39 of its 48 references that are in the map. First posted on arXiv 21 Mar 2023.

Builds on

39 of 48 references.

References · 39

  1. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,410
  2. 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,923
  3. 2016Pyramid Scene Parsing Networkcited by 2,472
  4. 2019Deep High-Resolution Representation Learning for Visual Recognitioncited by 1,417
  5. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,354
  6. 2022BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformerscited by 1,213
  7. 2019Habitat: A Platform for Embodied AI Researchcited by 1,163
  8. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,013
  9. 2019Segmentation Transformer: Object-Contextual Representations for Semantic Segmentationcited by 573
  10. 2022PETR: Position Embedding Transformation for Multi-View 3D Object Detectioncited by 550
  11. 2018Gibson Env: Real-World Perception for Embodied Agentscited by 477
  12. 2017Joint 2D-3D-Semantic Data for Indoor Scene Understandingcited by 404
27 more
  1. 2019Deep Multi-modal Object Detection and Semantic Segmentation for Autonomous Driving: Datasets, Methods, and Challengescited by 313
  2. 2022BEVFormer v2: Adapting Modern Image Backbones to Bird's-Eye-View Recognition via Perspective Supervisioncited by 302
  3. 2019Structured3D: A Large Photo-realistic Dataset for Structured 3D Modelingcited by 285
  4. 2022Cross-view Transformers for real-time Map-view Semantic Segmentationcited by 264
  5. 2019Gauge Equivariant Convolutional Networks and the Icosahedral CNNcited by 262
  6. 2022SegNeXt: Rethinking Convolutional Attention Design for Semantic Segmentationcited by 252
  7. 2017Computer Vision for Autonomous Vehicles: Problems, Datasets and State of the Artcited by 182
  8. 2019Cross-view Semantic Segmentation for Sensing Surroundingscited by 173
  9. 2022BEVSegFormer: Bird's Eye View Semantic Segmentation From Arbitrary Camera Rigscited by 123
  10. 2020HoHoNet: 360 Indoor Holistic Understanding with Latent Horizontal Featurescited by 104
  11. 2019Embodied Question Answering in Photorealistic Environments with Point Cloud Perceptioncited by 103
  12. 2018InteriorNet: Mega-scale Multi-sensor Photo-realistic Indoor Scenes Datasetcited by 87
  13. 2019Volumetric Instance-Aware Semantic Mapping and 3D Object Discoverycited by 79
  14. 2019Tangent Images for Mitigating Spherical Distortioncited by 79
  15. 2019Spherical CNNs on Unstructured Gridscited by 76
  16. 2022Bending Reality: Distortion-aware Transformers for Adapting to Panoramic Semantic Segmentationcited by 71
  17. 2020Semantic MapNet: Building Allocentric Semantic Maps and Representations from Egocentric Viewscited by 57
  18. 2019Orientation-aware Semantic Segmentation on Icosahedron Spherescited by 52
  19. 2019Spherical View Synthesis for Self-Supervised 360 Depth Estimationcited by 51
  20. 2020Spin-Weighted Spherical CNNscited by 50
  21. 2022Behind Every Domain There is a Shift: Adapting Distortion-aware Vision Transformers for Panoramic Semantic Segmentationcited by 46
  22. 2021Self-Supervised Pillar Motion Learning for Autonomous Drivingcited by 44
  23. 2021Pano3D: A Holistic Benchmark and a Solid Baseline for 360o Depth Estimationcited by 37
  24. 2021Trans4Trans: Efficient Transformer for Transparent Object Segmentation to Help Visually Impaired People Navigate in the Real Worldcited by 30
  25. 2020PillarFlow: End-to-end Birds-eye-view Flow Estimation for Autonomous Drivingcited by 16
  26. 2022Complementary Bi-directional Feature Compression for Indoor 360deg Semantic Segmentation with Self-distillationcited by 12
  27. 2022Trans4Map: Revisiting Holistic Bird's-Eye-View Mapping from Egocentric Images to Allocentric Semantics with Vision Transformerscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 6; 2025: 5; 2026: 6.

18 papers.

Sort:

By area:

  1. 2023CoBEV: Elevating Roadside 3D Object Detection with Depth and Height Complementaritycited by 18
  2. 2024SG-BEV: Satellite-Guided BEV Fusion for Cross-View Semantic Segmentationcited by 18
  3. 2024Open Panoramic Segmentationcited by 17
  4. 2025Omnidirectional Multi-Object Trackingcited by 12
  5. 2024Leveraging BEV Paradigm for Ground-to-Aerial Image Synthesiscited by 11
  6. 2024OneBEV: Using One Panoramic Image for Bird's-Eye-View Semantic Mappingcited by 8
  7. 2024Deformable Mamba for Wide Field of View Segmentationcited by 7
  8. 2024Occlusion-Aware Seamless Segmentationcited by 6
  9. 2026PanoSAMic: Panoramic Image Segmentation from SAM Feature Encoding and Dual View Fusioncited by 4
  10. 2025Dream to Recall: Imagination-Guided Experience Retrieval for Memory-Persistent Vision-and-Language Navigationcited by 3
  11. 2025KD360-VoxelBEV: LiDAR and 360-degree Camera Cross Modality Knowledge Distillation for Bird's-Eye-View Segmentationcited by 2
  12. 2026Panoramic Multimodal Semantic Occupancy Prediction for Quadruped Robotscited by 2
6 more
  1. 2025360-Degree Full-view Image Segmentation by Spherical Convolution compatible with Large-scale Planar Pre-trained Modelscited by 1
  2. 2026SO3UFormer: Learning Intrinsic Spherical Features for Rotation-Robust Panoramic Dense Predictioncited by 1
  3. 2026JOPP-3D: Joint Open Vocabulary Semantic Segmentation on Point Clouds and Panoramascited by 1
  4. 2026O3N: Omnidirectional Open-Vocabulary Occupancy Prediction for Embodied Intelligent Roboticscited by 1
  5. 2025Panoramic Out-of-Distribution Segmentationcited by 0
  6. 2026Faster Training, Fewer Labels: Self-Supervised Pretraining for Fine-Grained BEV Segmentationcited by 0