mlsanityData to 7 Oct 2026

Voxel Transformer for 3D Object Detection

arXiv 2109.02497 ↗First posted 6 Sep 2021cs.CVcited by 247

Not on the map. Why?

Abstract (arXiv)

We present Voxel Transformer (VoTr), a novel and effective voxel-based Transformer backbone for 3D object detection from point clouds. Conventional 3D convolutional backbones in voxel-based 3D detectors cannot efficiently capture large context information, which is crucial for object recognition and localization, owing to the limited receptive fields. In this paper, we resolve the problem by introducing a Transformer-based architecture that enables long-range relationships between voxels by self-attention. Given the fact that non-empty voxels are naturally sparse but numerous, directly applying standard Transformer on voxels is non-trivial. To this end, we propose the sparse voxel module and the submanifold voxel module, which can operate on the empty and non-empty voxel positions effectively. To further enlarge the attention range while maintaining comparable computational overhead to the convolutional counterparts, we propose two attention mechanisms for multi-head attention in those two modules: Local Attention and Dilated Attention, and we further propose Fast Voxel Query to accelerate the querying process in multi-head attention. VoTr contains a series of sparse and submanifold voxel modules and can be applied in most voxel-based detectors. Our proposed VoTr shows consistent improvement over the convolutional baselines while maintaining computational efficiency on the KITTI dataset and the Waymo Open dataset.

Key facts. Cited by 247 papers in the map (data through 7 Oct 2026). Builds on 39 of its 45 references that are in the map. First posted on arXiv 6 Sep 2021.

Builds on

39 of 45 references.

References · 39

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2020End-to-End Object Detection with Transformerscited by 7,384
  4. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
  5. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,384
  6. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,436
  7. 2018PointPillars: Fast Encoders for Object Detection from Point Cloudscited by 1,707
  8. 2020Big Bird: Transformers for Longer Sequencescited by 1,673
  9. 2020Point Transformercited by 1,462
  10. 2020Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformerscited by 1,368
  11. 2017VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detectioncited by 1,323
  12. 2020Center-based 3D Object Detection and Trackingcited by 1,143
27 more
  1. 2018PointRCNN: 3D Object Proposal Generation and Detection from Point Cloudcited by 939
  2. 2019PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detectioncited by 890
  3. 2016Multi-View 3D Object Detection Network for Autonomous Drivingcited by 721
  4. 2017Frustum PointNets for 3D Object Detection from RGB-D Datacited by 640
  5. 20203DSSD: Point-based 3D Single Stage Object Detectorcited by 440
  6. 2020Voxel R-CNN: Towards High Performance Voxel-based 3D Object Detectioncited by 409
  7. 2019From Points to Parts: 3D Object Detection from Point Cloud with Part-aware and Part-aggregation Networkcited by 366
  8. 2020MaX-DeepLab: End-to-End Panoptic Segmentation with Mask Transformerscited by 355
  9. 2020Point-GNN: Graph Neural Network for 3D Object Detection in a Point Cloudcited by 335
  10. 2017Joint 3D Proposal Generation and Object Detection from View Aggregationcited by 325
  11. 2019STD: Sparse-to-Dense 3D Object Detector for Point Cloudcited by 282
  12. 2020Deep Continuous Fusion for Multi-Sensor 3D Object Detectioncited by 263
  13. 2021One Million Scenes for Autonomous Driving: ONCE Datasetcited by 218
  14. 2020Multi-Task Multi-Sensor Fusion for 3D Object Detectioncited by 203
  15. 20203D Object Detection with Pointformercited by 194
  16. 20203D-CVF: Generating Joint Camera and LiDAR Features Using Cross-View Spatial Feature Fusion for 3D Object Detectioncited by 176
  17. 2019End-to-End Multi-View Fusion for 3D Object Detection in LiDAR Point Cloudscited by 171
  18. 2020CLOCs: Camera-LiDAR Object Candidates Fusion for 3D Object Detectioncited by 147
  19. 2020Pillar-based Object Detection for Autonomous Drivingcited by 137
  20. 2019LaserNet: An Efficient Probabilistic 3D Object Detector for Autonomous Drivingcited by 133
  21. 2019TANet: Robust 3D Object Detection from Point Clouds with Triple Attentioncited by 108
  22. 2019Interpolated Convolutional Networks for 3D Point Cloud Understandingcited by 86
  23. 2020HVNet: Hybrid Voxel Network for LiDAR Based 3D Object Detectioncited by 82
  24. 2020Range Conditioned Dilated Convolutions for Scale Invariant 3D Object Detectioncited by 78
  25. 2020AFDet: Anchor Free One Stage 3D Object Detectioncited by 63
  26. 2019StarNet: Targeted Computation for Object Detection in Point Cloudscited by 46
  27. 2019Patch Refinement -- Localized 3D Object Detectioncited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 10; 2022: 73; 2023: 71; 2024: 42; 2025: 34; 2026: 17.

247 papers.

Sort:

By area:

  1. 2022Perceiver-Actor: A Multi-Task Transformer for Robotic Manipulationcited by 545
  2. 2022TransFusion: Robust LiDAR-Camera Fusion for 3D Object Detection with Transformerscited by 464
  3. 2021AdaViT: Adaptive Vision Transformers for Efficient Image Recognitioncited by 228
  4. 2024GenAD: Generative End-to-End Autonomous Drivingcited by 225
  5. 2023VoxelNeXt: Fully Sparse VoxelNet for 3D Object Detection and Trackingcited by 221
  6. 2021Embracing Single Stride 3D Object Detector with Sparse Transformercited by 193
  7. 2022Stratified Transformer for 3D Point Cloud Segmentationcited by 166
  8. 2023A Language Agent for Autonomous Drivingcited by 161
  9. 2022Focal Sparse Convolutional Networks for 3D Object Detectioncited by 142
  10. 2024ShapeLLM: Universal 3D Object Understanding for Embodied Interactioncited by 133
  11. 20223D Object Detection for Autonomous Driving: A Comprehensive Surveycited by 132
  12. 2022Not All Points Are Equal: Learning Highly Efficient Point-based Detectors for 3D LiDAR Point Cloudscited by 131
38 more
  1. 2023Robo3D: Towards Robust and Reliable 3D Perception against Corruptionscited by 128
  2. 2022DeepInteraction: 3D Object Detection via Modality Interactioncited by 125
  3. 2023Spherical Transformer for LiDAR-based 3D Recognitioncited by 124
  4. 2022Delving into the Devils of Bird's-eye-view Perception: A Review, Evaluation and Recipecited by 122
  5. 2023DSVT: Dynamic Sparse Voxel Transformer with Rotated Setscited by 122
  6. 2021A Survey of Visual Transformerscited by 117
  7. 2023OctFormer: Octree-based Transformers for 3D Point Cloudscited by 105
  8. 2021Fast Point Transformercited by 103
  9. 2022Voxel Set Transformer: A Set-to-Set Approach to 3D Object Detection from Point Cloudscited by 102
  10. 2022CenterFormer: Center-based Transformer for 3D Object Detectioncited by 99
  11. 2022SWFormer: Sparse Window Transformer for 3D Object Detection in Point Cloudscited by 92
  12. 2022InvPT: Inverted Pyramid Multi-task Transformer for Dense Scene Understandingcited by 89
  13. 2021AFDetV2: Rethinking the Necessity of the Second Stage for Object Detection from Point Cloudscited by 88
  14. 2022Fully Sparse 3D Object Detectioncited by 84
  15. 2023SparseFusion: Fusing Multi-Modal Sparse Representations for Multi-Sensor 3D Object Detectioncited by 83
  16. 2022Sparse Fuse Dense: Towards High Quality 3D Detection with Depth Completioncited by 82
  17. 2023LoGoNet: Towards Accurate 3D Object Detection with Local-to-Global Cross-Modal Fusioncited by 80
  18. 2023Joint Token Pruning and Squeezing Towards More Aggressive Compression of Vision Transformerscited by 79
  19. 2023UniTR: A Unified and Efficient Multi-Modal Transformer for Bird's-Eye-View Representationcited by 74
  20. 2023FlatFormer: Flattened Window Attention for Efficient Point Cloud Transformercited by 72
  21. 2021PointMixer: MLP-Mixer for Point Cloud Understandingcited by 71
  22. 2022LargeKernel3D: Scaling up Kernels in 3D Sparse CNNscited by 71
  23. 2022Point Density-Aware Voxels for LiDAR 3D Object Detectioncited by 70
  24. 2024Voxel Mamba: Group-Free State Space Models for Point Cloud based 3D Object Detectioncited by 66
  25. 2022Occupancy-MAE: Self-supervised Pre-training Large-scale LiDAR Point Clouds with Masked Occupancy Autoencoderscited by 57
  26. 2022GD-MAE: Generative Decoder for MAE Pre-training on LiDAR Point Cloudscited by 57
  27. 2023PointOcc: Cylindrical Tri-Perspective View for Point-based 3D Semantic Occupancy Predictioncited by 51
  28. 2021EPNet++: Cascade Bi-directional Fusion for Multi-Modal 3D Object Detectioncited by 49
  29. 2022AutoAlignV2: Deformable Feature Aggregation for Dynamic Multi-Modal 3D Object Detectioncited by 46
  30. 2022NeRF-RPN: A general framework for object detection in NeRFscited by 46
  31. 2025SoFar: Language-Grounded Orientation Bridges Spatial Reasoning and Object Manipulationcited by 46
  32. 20223D Siamese Transformer Network for Single Object Tracking on Point Cloudscited by 44
  33. 2023UniDistill: A Universal Cross-Modality Knowledge Distillation Framework for 3D Object Detection in Bird's-Eye Viewcited by 43
  34. 2024OA-CNNs: Omni-Adaptive Sparse CNNs for 3D Semantic Segmentationcited by 43
  35. 2022Graph R-CNN: Towards Accurate 3D Object Detection with Semantic-Decorated Local Graphcited by 42
  36. 2022Delving into Sequential Patches for Deepfake Detectioncited by 41
  37. 2022Masked Autoencoder for Self-Supervised Pre-training on Lidar Point Cloudscited by 39
  38. 2022DID-M3D: Decoupling Instance Depth for Monocular 3D Object Detectioncited by 39

50 most cited of 247 All 247 (JSON)