mlsanityData to 14 Oct 2026

COTR: Compact Occupancy TRansformer for Vision-based 3D Occupancy Prediction

arXiv 2312.01919 ↗First posted 4 Dec 2023cs.CVcited by 96

Not on the map. Why?

Abstract (arXiv)

The autonomous driving community has shown significant interest in 3D occupancy prediction, driven by its exceptional geometric perception and general object recognition capabilities. To achieve this, current works try to construct a Tri-Perspective View (TPV) or Occupancy (OCC) representation extending from the Bird-Eye-View perception. However, compressed views like TPV representation lose 3D geometry information while raw and sparse OCC representation requires heavy but redundant computational costs. To address the above limitations, we propose Compact Occupancy TRansformer (COTR), with a geometry-aware occupancy encoder and a semantic-aware group decoder to reconstruct a compact 3D OCC representation. The occupancy encoder first generates a compact geometrical OCC feature through efficient explicit-implicit view transformation. Then, the occupancy decoder further enhances the semantic discriminability of the compact OCC representation by a coarse-to-fine semantic grouping strategy. Empirical experiments show that there are evident performance gains across multiple baselines, e.g., COTR outperforms baselines with a relative improvement of 8%-15%, demonstrating the superiority of our method.

Key facts. Cited by 96 papers in the map (data through 14 Oct 2026). Builds on 39 of its 41 references that are in the map. First posted on arXiv 4 Dec 2023.

Builds on

39 of 41 references.

References · 39

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  3. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  4. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
  5. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,451
  6. 2022BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformerscited by 1,218
  7. 2021Per-Pixel Classification is Not All You Need for Semantic Segmentationcited by 1,152
  8. 2020Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3Dcited by 1,044
  9. 2021BEVDet: High-performance Multi-camera 3D Object Detection in Bird-Eye-Viewcited by 713
  10. 2021DETR3D: 3D Object Detection from Multi-view Images via 3D-to-2D Queriescited by 658
  11. 2022BEVDepth: Acquisition of Reliable Depth for Multi-view 3D Object Detectioncited by 572
  12. 2022PETR: Position Embedding Transformation for Multi-View 3D Object Detectioncited by 552
27 more
  1. 2016Semantic Scene Completion from a Single Depth Imagecited by 472
  2. 2023Tri-Perspective View for Vision-Based 3D Semantic Occupancy Predictioncited by 409
  3. 2021MonoScene: Monocular 3D Semantic Scene Completioncited by 363
  4. 2022BEVDet4D: Exploit Temporal Cues in Multi-camera 3D Object Detectioncited by 341
  5. 2022BEVFusion: A Simple and Robust LiDAR-Camera Fusion Frameworkcited by 336
  6. 2023Occ3D: A Large-Scale 3D Occupancy Prediction Benchmark for Autonomous Drivingcited by 332
  7. 2021Categorical Depth Distribution Network for Monocular 3D Object Detectioncited by 327
  8. 2023SurroundOcc: Multi-Camera 3D Occupancy Prediction for Autonomous Drivingcited by 316
  9. 2023VoxFormer: Sparse Voxel Transformer for Camera-based 3D Semantic Scene Completioncited by 313
  10. 2022BEVFormer v2: Adapting Modern Image Backbones to Bird's-Eye-View Recognition via Perspective Supervisioncited by 302
  11. 2023OccFormer: Dual-path Transformer for Vision-based 3D Semantic Occupancy Predictioncited by 293
  12. 2020Sparse Single Sweep LiDAR Point Cloud Segmentation via Learning Contextual Shape Priors from Scene Completioncited by 223
  13. 2023OpenOccupancy: A Large Scale Benchmark for Surrounding Semantic Occupancy Perceptioncited by 221
  14. 2023Scene as Occupancycited by 209
  15. 2020LMSCNet: Lightweight Multiscale 3D Semantic Completioncited by 188
  16. 2020Learning From Multiple Experts: Self-paced Knowledge Distillation for Long-tailed Classificationcited by 175
  17. 2020Overcoming Classifier Imbalance for Long-tail Object Detection with Balanced Group Softmaxcited by 152
  18. 2022PolarFormer: Multi-camera 3D Object Detection with Polar Transformercited by 147
  19. 2020Anisotropic Convolutional Networks for 3D Semantic Scene Completioncited by 127
  20. 20203D Sketch-aware Semantic Scene Completion via Semi-supervised Structure Priorcited by 126
  21. 2022Delving into the Devils of Bird's-eye-view Perception: A Review, Evaluation and Recipecited by 122
  22. 2022Group DETR: Fast DETR Training with Group-Wise One-to-Many Assignmentcited by 98
  23. 2023FB-BEV: BEV Representation from Forward-Backward View Transformationscited by 98
  24. 2021ACE: Ally Complementary Experts for Solving Long-Tailed Recognition in One-Shotcited by 86
  25. 2023OccDepth: A Depth-Aware Method for 3D Semantic Scene Completioncited by 86
  26. 2018Monocular Semantic Occupancy Grid Mapping with Convolutional Variational Encoder-Decoder Networkscited by 84
  27. 2023FrustumFormer: Adaptive Instance-aware Resampling for Multi-view 3D Detectioncited by 21

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 21; 2025: 40; 2026: 34.

96 papers.

Sort:

By area:

  1. 2024GaussianWorld: Gaussian World Model for Streaming 3D Occupancy Predictioncited by 67
  2. 2024OPUS: Occupancy Prediction Using a Sparse Setcited by 56
  3. 2024DrivingForward: Feed-forward 3D Gaussian Splatting for Driving Scene Reconstruction from Flexible Surround-view Inputcited by 53
  4. 2023OctreeOcc: Efficient and Multi-Granularity Occupancy Prediction Using Octree Queriescited by 48
  5. 2024GaussTR: Foundation Model-Aligned Gaussian Transformer for Self-Supervised 3D Spatial Understandingcited by 46
  6. 2024A Survey on Occupancy Perception for Autonomous Driving: The Information Fusion Perspectivecited by 37
  7. 2024RadarOcc: Robust 3D Occupancy Prediction with 4D Imaging Radarcited by 33
  8. 2024OccMamba: Semantic Occupancy Prediction with State Space Modelscited by 30
  9. 2024Vision-based 3D occupancy prediction in autonomous driving: a review and outlookcited by 29
  10. 2025UniOcc: A Unified Benchmark for Occupancy Forecasting and Prediction in Autonomous Drivingcited by 28
  11. 2025GaussianFlowOcc: Sparse and Weakly Supervised Occupancy Estimation using Gaussian Splatting and Temporal Flowcited by 24
  12. 2025STCOcc: Sparse Spatial-Temporal Cascade Renovation for 3D Occupancy and Scene Flow Predictioncited by 23
38 more
  1. 2024GEOcc: Geometrically Enhanced 3D Occupancy Network with Implicit-Explicit Depth Fusion and Contextual Self-Supervisioncited by 21
  2. 2024ALOcc: Adaptive Lifting-Based 3D Semantic Occupancy and Cost Volume-Based Flow Predictionscited by 20
  3. 2025QuadricFormer: Scene as Superquadrics for 3D Semantic Occupancy Predictioncited by 20
  4. 2025GaussRender: Learning 3D Occupancy with Gaussian Renderingcited by 18
  5. 2024Language Driven Occupancy Predictioncited by 17
  6. 2024ProtoOcc: Accurate, Efficient 3D Occupancy Prediction Using Dual Branch Encoder-Prototype Query Decodercited by 17
  7. 2024Deep Height Decoupling for Precise Vision-based 3D Occupancy Predictioncited by 16
  8. 2024DAOcc: 3D Object Detection Assisted Multi-Sensor Fusion for 3D Occupancy Predictioncited by 15
  9. 2026ResWorld: Temporal Residual World Model for End-to-End Autonomous Drivingcited by 15
  10. 2025GaussianFormer3D: Multi-Modal Gaussian-based Semantic Occupancy Prediction with 3D Deformable Attentioncited by 14
  11. 2024Offboard Occupancy Refinement with Hybrid Propagation for Autonomous Drivingcited by 12
  12. 2025OccProphet: Pushing Efficiency Frontier of Camera-Only 4D Occupancy Forecasting with Observer-Forecaster-Refiner Frameworkcited by 12
  13. 2025SparseWorld: A Flexible, Adaptive, and Efficient 4D Occupancy World Model Powered by Sparse and Dynamic Queriescited by 11
  14. 2025VoxelSplat: Dynamic Gaussian Splatting as an Effective Loss for Occupancy and Flow Predictioncited by 10
  15. 20253D Occupancy Prediction with Low-Resolution Queries via Prototype-aware View Transformationcited by 9
  16. 2025Rethinking Temporal Fusion with a Unified Gradient Descent View for 3D Semantic Occupancy Predictioncited by 9
  17. 2025Occupancy Learning with Spatiotemporal Memorycited by 9
  18. 2025OccTENS: 3D Occupancy World Model via Temporal Next-Scale Predictioncited by 9
  19. 2025Semantic Causality-Aware Vision-Based 3D Occupancy Predictioncited by 8
  20. 2024Robust 3D Semantic Occupancy Prediction with Calibration-free Spatial Transformationcited by 7
  21. 2026ForecastOcc: Vision-based Semantic Occupancy Forecastingcited by 7
  22. 2024LightOcc: Lightweight Spatial Embedding for Efficient Vision-based 3D Occupancy Predictioncited by 6
  23. 2025HENet++: Hybrid Encoding and Multi-task Learning for 3D Perception and End-to-end Autonomous Drivingcited by 6
  24. 2025ShelfOcc: Native 3D Supervision beyond LiDAR for Vision-Based Occupancy Estimationcited by 6
  25. 2026OccAny: Generalized Unconstrained Urban 3D Occupancycited by 6
  26. 2024MambaOcc: Visual State Space Model for BEV-based Occupancy Prediction with Local Adaptive Reorderingcited by 5
  27. 2024SignEye: Traffic Sign Interpretation from Vehicle First-Person Viewcited by 5
  28. 2026GaussianOcc3D: A Gaussian-Based Adaptive Multi-modal 3D Occupancy Predictioncited by 5
  29. 2026Panoramic Multimodal Semantic Occupancy Prediction for Quadruped Robotscited by 5
  30. 2025Inverse++: Vision-Centric 3D Semantic Occupancy Prediction Assisted with 3D Object Detectioncited by 4
  31. 2025Progressive Gaussian Transformer with Anisotropy-aware Sampling for Open Vocabulary Occupancy Predictioncited by 4
  32. 2026SparseOccVLA: Bridging Occupancy and Vision-Language Models via Sparse Queries for Unified 4D Scene Understanding and Planningcited by 4
  33. 2025Event-aided Semantic Scene Completioncited by 3
  34. 2025TrackOcc: Camera-based 4D Panoptic Occupancy Trackingcited by 3
  35. 2025FMLGS: Fast Multilevel Language Embedded Gaussians for Part-level Interactive Agentscited by 3
  36. 2025See through the Dark: Learning Illumination-affined Representations for Nighttime Occupancy Predictioncited by 3
  37. 2026Latent Gaussian Splatting for 4D Panoptic Occupancy Trackingcited by 3
  38. 2025Camera-Only 3D Panoptic Scene Completion for Autonomous Driving through Differentiable Object Shapescited by 2

50 most cited of 96 All 96 (JSON)