mlsanityData to 7 Oct 2026

ET-Former: Efficient Triplane Deformable Attention for 3D Semantic Scene Completion From Monocular Camera

arXiv 2410.11019 ↗First posted 14 Oct 2024cs.CVcited by 3

Not on the map. Why?

Abstract (arXiv)

We introduce ET-Former, a novel end-to-end algorithm for semantic scene completion using a single monocular camera. Our approach generates a semantic occupancy map from single RGB observation while simultaneously providing uncertainty estimates for semantic predictions. By designing a triplane-based deformable attention mechanism, our approach improves geometric understanding of the scene than other SOTA approaches and reduces noise in semantic predictions. Additionally, through the use of a Conditional Variational AutoEncoder (CVAE), we estimate the uncertainties of these predictions. The generated semantic and uncertainty maps will help formulate navigation strategies that facilitate safe and permissible decision making in the future. Evaluated on the Semantic-KITTI dataset, ET-Former achieves the highest Intersection over Union (IoU) and mean IoU (mIoU) scores while maintaining the lowest GPU memory usage, surpassing state-of-the-art (SOTA) methods. It improves the SOTA scores of IoU from 44.71 to 51.49 and mIoU from 15.04 to 16.30 on SeamnticKITTI test, with a notably low training memory consumption of 10.9 GB. Project page: https://github.com/jingGM/ET-Former.git.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 37 of its 47 references that are in the map. First posted on arXiv 14 Oct 2024.

Builds on

37 of 47 references.

References · 37

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2016Feature Pyramid Networks for Object Detectioncited by 5,021
  3. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,410
  4. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,682
  5. 2022DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detectioncited by 1,306
  6. 2022BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformerscited by 1,213
  7. 2020Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3Dcited by 1,040
  8. 2021BEVDet: High-performance Multi-camera 3D Object Detection in Bird-Eye-Viewcited by 711
  9. 2021KiloNeRF: Speeding up Neural Radiance Fields with Thousands of Tiny MLPscited by 672
  10. 2020Learning Continuous Image Representation with Local Implicit Image Functioncited by 550
  11. 2016Semantic Scene Completion from a Single Depth Imagecited by 468
  12. 2023Tri-Perspective View for Vision-Based 3D Semantic Occupancy Predictioncited by 404
25 more
  1. 2021MonoScene: Monocular 3D Semantic Scene Completioncited by 357
  2. 2021Categorical Depth Distribution Network for Monocular 3D Object Detectioncited by 326
  3. 2023VoxFormer: Sparse Voxel Transformer for Camera-based 3D Semantic Scene Completioncited by 306
  4. 2023OccFormer: Dual-path Transformer for Vision-based 3D Semantic Occupancy Predictioncited by 287
  5. 20223D Neural Field Generation using Triplane Diffusioncited by 274
  6. 2022Cross-view Transformers for real-time Map-view Semantic Segmentationcited by 264
  7. 2022BEVerse: Unified Perception and Prediction in Birds-Eye-View for Vision-Centric Autonomous Drivingcited by 191
  8. 2020LMSCNet: Lightweight Multiscale 3D Semantic Completioncited by 187
  9. 2024GaussianFormer: Scene as Gaussians for Vision-Based 3D Semantic Occupancy Predictioncited by 155
  10. 2022PolarFormer: Multi-camera 3D Object Detection with Polar Transformercited by 147
  11. 20203D Sketch-aware Semantic Scene Completion via Semi-supervised Structure Priorcited by 124
  12. 2021Holistic 3D Scene Understanding from a Single Image with Implicit Representationcited by 94
  13. 2023Symphonize 3D Semantic Scene Completion with Contextual Instance Queriescited by 88
  14. 2019Image-based 3D Object Reconstruction: State-of-the-Art and Trends in the Deep Learning Eracited by 87
  15. 2024SGS-SLAM: Semantic Gaussian Splatting For Neural Dense SLAMcited by 86
  16. 2021MobileStereoNet: Towards Lightweight Deep Networks for Stereo Matchingcited by 82
  17. 2023SCPNet: Semantic Scene Completion on Point Cloudcited by 80
  18. 2019SG-NN: Sparse Generative Neural Networks for Self-Supervised Scene Completion of RGB-D Scanscited by 77
  19. 20213D Semantic Scene Completion: a Surveycited by 74
  20. 2023OctreeOcc: Efficient and Multi-Granularity Occupancy Prediction Using Octree Queriescited by 48
  21. 2020CoReNet: Coherent 3D scene reconstruction from a single RGB imagecited by 45
  22. 2021VolumeFusion: Deep Depth Fusion for 3D Scene Reconstructioncited by 44
  23. 2024MonoOcc: Digging into Monocular Semantic Occupancy Predictioncited by 43
  24. 2023LODE: Locally Conditioned Eikonal Implicit Scene Completion from Sparse LiDARcited by 31
  25. 2021Conditional Image Generation by Conditioning Variational Auto-Encoderscited by 25

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026FlowSSC: Universal Generative Monocular Semantic Scene Completion via One-Step Latent Diffusioncited by 2
  2. 2026GaussianSSC: Triplane-Guided Directional Gaussian Fields for 3D Semantic Completioncited by 0
  3. 2026Sparsity-Aware Voxel Attention and Foreground Modulation for 3D Semantic Scene Completioncited by 0