mlsanityData to 7 Oct 2026

Tri-Perspective View for Vision-Based 3D Semantic Occupancy Prediction

arXiv 2302.07817 ↗First posted 15 Feb 2023cs.CV cs.AI cs.LGcited by 404

Not on the map. Why?

Cited by 32 in World models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Modern methods for vision-centric autonomous driving perception widely adopt the bird's-eye-view (BEV) representation to describe a 3D scene. Despite its better efficiency than voxel representation, it has difficulty describing the fine-grained 3D structure of a scene with a single plane. To address this, we propose a tri-perspective view (TPV) representation which accompanies BEV with two additional perpendicular planes. We model each point in the 3D space by summing its projected features on the three planes. To lift image features to the 3D TPV space, we further propose a transformer-based TPV encoder (TPVFormer) to obtain the TPV features effectively. We employ the attention mechanism to aggregate the image features corresponding to each query in each TPV plane. Experiments show that our model trained with sparse supervision effectively predicts the semantic occupancy for all voxels. We demonstrate for the first time that using only camera inputs can achieve comparable performance with LiDAR-based methods on the LiDAR segmentation task on nuScenes. Code: https://github.com/wzzheng/TPVFormer.

Key facts. Cited by 404 papers in the map (data through 7 Oct 2026). Builds on 47 of its 57 references that are in the map. First posted on arXiv 15 Feb 2023.

Builds on

47 of 57 references.

References · 47

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  3. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  4. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,410
  5. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,682
  6. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,037
  7. 2018PointPillars: Fast Encoders for Object Detection from Point Cloudscited by 1,707
  8. 2017Deformable Convolutional Networkscited by 1,481
  9. 2017VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detectioncited by 1,323
  10. 2022TensoRF: Tensorial Radiance Fieldscited by 1,275
  11. 2022BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformerscited by 1,213
  12. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,204
35 more
  1. 20194D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networkscited by 1,201
  2. 2020Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3Dcited by 1,040
  3. 2022BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representationcited by 844
  4. 2021BEVDet: High-performance Multi-camera 3D Object Detection in Bird-Eye-Viewcited by 711
  5. 2021KiloNeRF: Speeding up Neural Radiance Fields with Thousands of Tiny MLPscited by 672
  6. 2022BEVDepth: Acquisition of Reliable Depth for Multi-view 3D Object Detectioncited by 568
  7. 2020Learning Continuous Image Representation with Local Implicit Image Functioncited by 550
  8. 2022PETR: Position Embedding Transformation for Multi-View 3D Object Detectioncited by 550
  9. 2021FCOS3D: Fully Convolutional One-Stage Monocular 3D Object Detectioncited by 444
  10. 2020Searching Efficient 3D Architectures with Sparse Point-Voxel Convolutioncited by 422
  11. 2020Cylindrical and Asymmetrical 3D Convolution Networks for LiDAR Segmentationcited by 388
  12. 2021MonoScene: Monocular 3D Semantic Scene Completioncited by 357
  13. 2022BEVFusion: A Simple and Robust LiDAR-Camera Fusion Frameworkcited by 334
  14. 2021Categorical Depth Distribution Network for Monocular 3D Object Detectioncited by 326
  15. 2020Deep Local Shapes: Learning Local SDF Priors for Detailed 3D Reconstructioncited by 321
  16. 2020PolarNet: An Improved Grid Representation for Online LiDAR Point Clouds Semantic Segmentationcited by 274
  17. 2021FIERY: Future Instance Prediction in Bird's-Eye View from Surround Monocular Camerascited by 239
  18. 2020SalsaNext: Fast, Uncertainty-aware Semantic Segmentation of LiDAR Point Clouds for Autonomous Drivingcited by 226
  19. 2020Sparse Single Sweep LiDAR Point Cloud Segmentation via Learning Contextual Shape Priors from Scene Completioncited by 222
  20. 2022Unifying Voxel-based Representation with Transformer for 3D Object Detectioncited by 202
  21. 2022BEVerse: Unified Perception and Prediction in Birds-Eye-View for Vision-Centric Autonomous Drivingcited by 191
  22. 2020LMSCNet: Lightweight Multiscale 3D Semantic Completioncited by 187
  23. 2021Panoptic nuScenes: A Large-Scale Benchmark for LiDAR Panoptic Segmentation and Trackingcited by 165
  24. 2021(AF)2-S3Net: Attentive Feature Fusion with Adaptive Feature Selection for Sparse Semantic Segmentation Networkcited by 164
  25. 2022PolarFormer: Multi-camera 3D Object Detection with Polar Transformercited by 147
  26. 2020Anisotropic Convolutional Networks for 3D Semantic Scene Completioncited by 126
  27. 20203D Sketch-aware Semantic Scene Completion via Semi-supervised Structure Priorcited by 124
  28. 2020AMVNet: Assertion-based Multi-View Fusion Network for LiDAR Semantic Segmentationcited by 87
  29. 2022SurroundDepth: Entangling Surrounding Views for Self-Supervised Multi-Camera Depth Estimationcited by 76
  30. 2022LidarMultiNet: Towards a Unified Multi-Task Network for LiDAR Perceptioncited by 74
  31. 2022StretchBEV: Stretching Future Instance Prediction Spatially and Temporallycited by 47
  32. 2021Full Surround Monodepth from Multiple Camerascited by 41
  33. 2020Multi-scale Interaction for Real-time LiDAR Data Segmentation on an Embedded Platformcited by 40
  34. 2021DRINet++: Efficient Voxel-as-point Point Cloud Segmentationcited by 23
  35. 2022A Simple Baseline for Multi-Camera 3D Object Detectioncited by 16

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 53; 2024: 129; 2025: 137; 2026: 85.

404 papers.

Sort:

By area:

  1. 2023Occ3D: A Large-Scale 3D Occupancy Prediction Benchmark for Autonomous Drivingcited by 325
  2. 2023SurroundOcc: Multi-Camera 3D Occupancy Prediction for Autonomous Drivingcited by 310
  3. 2023OccFormer: Dual-path Transformer for Vision-based 3D Semantic Occupancy Predictioncited by 287
  4. 2023OccWorld: Learning a 3D Occupancy World Model for Autonomous Drivingcited by 230
  5. 2023Michelangelo: Conditional 3D Shape Generation based on Shape-Image-Text Aligned Latent Representationcited by 225
  6. 2024GenAD: Generative End-to-End Autonomous Drivingcited by 225
  7. 2023OpenOccupancy: A Large Scale Benchmark for Surrounding Semantic Occupancy Perceptioncited by 218
  8. 2023Scene as Occupancycited by 208
  9. 2024Enhancing End-to-End Autonomous Driving with Latent World Modelcited by 161
  10. 2024GaussianFormer: Scene as Gaussians for Vision-Based 3D Semantic Occupancy Predictioncited by 155
  11. 2025Streaming 4D Visual Geometry Transformercited by 147
  12. 2023EmbodiedScan: A Holistic Multi-Modal 3D Perception Suite Towards Embodied AIcited by 146
38 more
  1. 2023SelfOcc: Self-Supervised Vision-Based 3D Occupancy Predictioncited by 133
  2. 2023RenderOcc: Vision-Centric 3D Occupancy Prediction with 2D Rendering Supervisioncited by 126
  3. 2023PanoOcc: Unified Occupancy Representation for Camera-based 3D Panoptic Segmentationcited by 114
  4. 2024textitS3Gaussian: Self-Supervised Street Gaussians for Autonomous Drivingcited by 105
  5. 2024UniScene: Unified Occupancy-centric Driving Scene Generationcited by 102
  6. 2023FlashOcc: Fast and Memory-Efficient Occupancy Prediction via Channel-to-Height Plugincited by 101
  7. 2023Visual Point Cloud Forecasting enables Scalable Autonomous Drivingcited by 97
  8. 2023COTR: Compact Occupancy TRansformer for Vision-based 3D Occupancy Predictioncited by 94
  9. 2024SparseOcc: Rethinking Sparse Latent Representation for Vision-Based Semantic Occupancy Predictioncited by 94
  10. 2023Symphonize 3D Semantic Scene Completion with Contextual Instance Queriescited by 88
  11. 2024GaussianFormer-2: Probabilistic Gaussian Superposition for Efficient 3D Occupancy Predictioncited by 83
  12. 2025Point3R: Streaming 3D Reconstruction with Explicit Spatial Pointer Memorycited by 82
  13. 2024OccSora: 4D Occupancy Generation Models as World Simulators for Autonomous Drivingcited by 76
  14. 2023SSCBench: A Large-Scale 3D Semantic Scene Completion Benchmark for Autonomous Drivingcited by 72
  15. 2024Driving in the Occupancy World: Vision-Centric 4D Occupancy Forecasting and Planning via World Models for Autonomous Drivingcited by 67
  16. 2024GaussianWorld: Gaussian World Model for Streaming 3D Occupancy Predictioncited by 66
  17. 2024Volumetric Environment Representation for Vision-Language Navigationcited by 63
  18. 2023Fully Sparse 3D Occupancy Predictioncited by 61
  19. 2025A Survey of World Models for Autonomous Drivingcited by 60
  20. 2024DOME: Taming Diffusion Model into High-Fidelity Controllable Occupancy World Modelcited by 58
  21. 2024OPUS: Occupancy Prediction Using a Sparse Setcited by 56
  22. 2024SCube: Instant Large-Scale Scene Reconstruction using VoxSplatscited by 56
  23. 2024POP-3D: Open-Vocabulary 3D Occupancy Prediction from Imagescited by 52
  24. 2023PointOcc: Cylindrical Tri-Perspective View for Point-based 3D Semantic Occupancy Predictioncited by 51
  25. 2024FastOcc: Accelerating 3D Occupancy Prediction by Fusing the 2D Bird's-Eye View and Perspective Viewcited by 51
  26. 2023Cam4DOcc: Benchmark for Camera-Only 4D Occupancy Forecasting in Autonomous Driving Applicationscited by 50
  27. 2024Context and Geometry Aware Voxel Transformer for Semantic Scene Completioncited by 50
  28. 2024GaussianOcc: Fully Self-supervised and Efficient 3D Occupancy Estimation with Gaussian Splattingcited by 50
  29. 2023OctreeOcc: Efficient and Multi-Granularity Occupancy Prediction Using Octree Queriescited by 48
  30. 2024Co-Occ: Coupling Explicit Feature Fusion with Volume Rendering Regularization for Multi-Modal 3D Semantic Occupancy Predictioncited by 46
  31. 2024Navigation-Guided Sparse Scene Representation for End-to-End Autonomous Drivingcited by 46
  32. 2024GaussTR: Foundation Model-Aligned Gaussian Transformer for Self-Supervised 3D Spatial Understandingcited by 46
  33. 2024RenderWorld: World Model with Self-Supervised 3D Labelcited by 44
  34. 2024MonoOcc: Digging into Monocular Semantic Occupancy Predictioncited by 43
  35. 2024Not All Voxels Are Equal: Hardness-Aware Semantic Scene Completion with Self-Distillationcited by 43
  36. 2024EmbodiedOcc: Embodied 3D Occupancy Prediction for Vision-based Online Scene Understandingcited by 43
  37. 2023PaSCo: Urban 3D Panoptic Scene Completion with Uncertainty Awarenesscited by 42
  38. 2024Tri-Perspective View Decomposition for Geometry-Aware Depth Completioncited by 42

50 most cited of 404 All 404 (JSON)