mlsanityData to 14 Oct 2026

M3DeTR: Multi-representation, Multi-scale, Mutual-relation 3D Object Detection with Transformers

arXiv 2104.11896 ↗First posted 24 Apr 2021cs.CVcited by 74

Not on the map. Why?

Abstract (arXiv)

We present a novel architecture for 3D object detection, M3DeTR, which combines different point cloud representations (raw, voxels, bird-eye view) with different feature scales based on multi-scale feature pyramids. M3DeTR is the first approach that unifies multiple point cloud representations, feature scales, as well as models mutual relationships between point clouds simultaneously using transformers. We perform extensive ablation experiments that highlight the benefits of fusing representation and scale, and modeling the relationships. Our method achieves state-of-the-art performance on the KITTI 3D object detection dataset and Waymo Open Dataset. Results show that M3DeTR improves the baseline significantly by 1.48% mAP for all classes on Waymo Open Dataset. In particular, our approach ranks 1st on the well-known KITTI 3D Detection Benchmark for both car and cyclist classes, and ranks 1st on Waymo Open Dataset with single frame point cloud input. Our code is available at: https://github.com/rayguan97/M3DETR.

Key facts. Cited by 74 papers in the map (data through 14 Oct 2026). Builds on 57 of its 65 references that are in the map. First posted on arXiv 24 Apr 2021.

Builds on

57 of 65 references.

References · 57

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
  5. 2020End-to-End Object Detection with Transformerscited by 7,414
  6. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,248
  7. 2016Feature Pyramid Networks for Object Detectioncited by 5,036
  8. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
  9. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,401
  10. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,422
  11. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,451
  12. 2018Dynamic Graph CNN for Learning on Point Cloudscited by 2,223
38 more
  1. 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,222
  2. 2013Rich feature hierarchies for accurate object detection and semantic segmentationcited by 2,162
  3. 2018PointPillars: Fast Encoders for Object Detection from Point Cloudscited by 1,715
  4. 2019LXMERT: Learning Cross-Modality Encoder Representations from Transformerscited by 1,553
  5. 2020Point Transformercited by 1,470
  6. 2017VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detectioncited by 1,323
  7. 2019KPConv: Flexible and Deformable Convolution for Point Cloudscited by 1,205
  8. 2020Center-based 3D Object Detection and Trackingcited by 1,144
  9. 2019VL-BERT: Pre-training of Generic Visual-Linguistic Representationscited by 954
  10. 2018PointRCNN: 3D Object Proposal Generation and Detection from Point Cloudcited by 941
  11. 2019PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detectioncited by 891
  12. 2017Focal Loss for Dense Object Detectioncited by 752
  13. 2020PCT: Point cloud transformercited by 732
  14. 2016Multi-View 3D Object Detection Network for Autonomous Drivingcited by 724
  15. 2017Frustum PointNets for 3D Object Detection from RGB-D Datacited by 640
  16. 2019PointPainting: Sequential Fusion for 3D Object Detectioncited by 449
  17. 20203DSSD: Point-based 3D Single Stage Object Detectorcited by 440
  18. 2020Searching Efficient 3D Architectures with Sparse Point-Voxel Convolutioncited by 423
  19. 2019From Points to Parts: 3D Object Detection from Point Cloud with Part-aware and Part-aggregation Networkcited by 368
  20. 2019PIXOR: Real-time 3D Object Detection from Point Cloudscited by 358
  21. 2019Class-balanced Grouping and Sampling for Point Cloud 3D Object Detectioncited by 350
  22. 2020Learning Texture Transformer Network for Image Super-Resolutioncited by 347
  23. 2019Point-Voxel CNN for Efficient 3D Deep Learningcited by 343
  24. 2017Joint 3D Proposal Generation and Object Detection from View Aggregationcited by 326
  25. 2019STD: Sparse-to-Dense 3D Object Detector for Point Cloudcited by 282
  26. 2020Deep Continuous Fusion for Multi-Sensor 3D Object Detectioncited by 263
  27. 2018A disciplined approach to neural network hyper-parameters: Part 1 -- learning rate, batch size, momentum, and weight decaycited by 226
  28. 2020Multi-Task Multi-Sensor Fusion for 3D Object Detectioncited by 203
  29. 20203D Object Detection with Pointformercited by 194
  30. 2020Learning Joint Spatial-Temporal Transformations for Video Inpaintingcited by 163
  31. 2021RangeDet:In Defense of Range View for LiDAR-based 3D Object Detectioncited by 140
  32. 2019LaserNet: An Efficient Probabilistic 3D Object Detector for Autonomous Drivingcited by 133
  33. 2019Fast Point R-CNNcited by 119
  34. 2019Iterative Answer Prediction with Pointer-Augmented Multimodal Transformers for TextVQAcited by 107
  35. 2019Object as Hotspots: An Anchor-Free 3D Object Detection Approach via Firing of Hotspotscited by 84
  36. 2020HVNet: Hybrid Voxel Network for LiDAR Based 3D Object Detectioncited by 82
  37. 2020Range Conditioned Dilated Convolutions for Scale Invariant 3D Object Detectioncited by 78
  38. 2019What You See is What You Get: Exploiting Visibility for 3D Object Detectioncited by 69

50 most cited of 57 All 57 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 4; 2022: 20; 2023: 25; 2024: 11; 2025: 11; 2026: 3.

74 papers.

Sort:

By area:

  1. 2023VoxelNeXt: Fully Sparse VoxelNet for 3D Object Detection and Trackingcited by 222
  2. 2021Multimodal Virtual Point 3D Detectioncited by 166
  3. 2022PersFormer: 3D Lane Detection via Perspective Transformer and the OpenLane Benchmarkcited by 140
  4. 20223D Object Detection for Autonomous Driving: A Comprehensive Surveycited by 132
  5. 2023DSVT: Dynamic Sparse Voxel Transformer with Rotated Setscited by 122
  6. 2022CenterFormer: Center-based Transformer for 3D Object Detectioncited by 99
  7. 2021AFDetV2: Rethinking the Necessity of the Second Stage for Object Detection from Point Cloudscited by 88
  8. 2022Fully Sparse 3D Object Detectioncited by 84
  9. 2023LoGoNet: Towards Accurate 3D Object Detection with Local-to-Global Cross-Modal Fusioncited by 80
  10. 2023FlatFormer: Flattened Window Attention for Efficient Point Cloud Transformercited by 73
  11. 2022LargeKernel3D: Scaling up Kernels in 3D Sparse CNNscited by 72
  12. 2023A Comprehensive Survey on Applications of Transformers for Deep Learning Taskscited by 71
38 more
  1. 2024Voxel Mamba: Group-Free State Space Models for Point Cloud based 3D Object Detectioncited by 66
  2. 2021Multi-Modal 3D Object Detection in Autonomous Driving: a Surveycited by 63
  3. 2022GD-MAE: Generative Decoder for MAE Pre-training on LiDAR Point Cloudscited by 57
  4. 2021EPNet++: Cascade Bi-directional Fusion for Multi-Modal 3D Object Detectioncited by 49
  5. 2024LION: Linear Group RNN for 3D Object Detection in Point Cloudscited by 38
  6. 2023FSD V2: Improving Fully Sparse 3D Object Detection with Virtual Voxelscited by 34
  7. 2022CAT-Det: Contrastively Augmented Transformer for Multi-modal 3D Object Detectioncited by 32
  8. 2023Super Sparse 3D Object Detectioncited by 32
  9. 2023DetZero: Rethinking Offboard 3D Object Detection with Long-term Sequential Point Cloudscited by 31
  10. 2022Homogeneous Multi-modal Feature Fusion and Interaction for 3D Object Detectioncited by 26
  11. 20223D Vision with Transformers: A Surveycited by 21
  12. 2022LCPFormer: Towards Effective 3D Point Cloud Analysis via Local Context Propagation in Transformerscited by 21
  13. 2023PVT-SSD: Single-Stage 3D Object Detector with Point-Voxel Transformercited by 21
  14. 2024SEED: A Simple and Effective 3D DETR in Point Cloudscited by 19
  15. 2022Few-Shot Learning Meets Transformer: Unified Query-Support Transformers for Few-Shot Classificationcited by 16
  16. 2023MSF: Motion-guided Sequential Fusion for Efficient 3D Object Detection from Point Cloud Sequencescited by 15
  17. 2022INT: Towards Infinite-frames 3D Detection with An Efficient Frameworkcited by 14
  18. 2023MS3D++: Ensemble of Experts for Multi-Source Unsupervised Domain Adaptation in 3D Object Detectioncited by 14
  19. 2022OccAM's Laser: Occlusion-based Attribution Maps for 3D Object Detectors on LiDAR Datacited by 13
  20. 2022TAG: Boosting Text-VQA via Text-aware Visual Question-answer Generationcited by 12
  21. 2023LiDARFormer: A Unified Transformer-based Multi-task Network for LiDAR Perceptioncited by 11
  22. 2024ScatterFormer: Efficient Voxel Transformer with Scattered Linear Attentioncited by 11
  23. 2024An Empirical Study of the Generalization Ability of Lidar 3D Object Detectors to Unseen Domainscited by 10
  24. 2022Can We Solve 3D Vision Tasks Starting from A 2D Vision Transformer?cited by 8
  25. 2022LidarAugment: Searching for Scalable 3D LiDAR Data Augmentationscited by 8
  26. 2023Transformer-Based Autonomous Driving Models and Deployment-Oriented Compression: A Surveycited by 8
  27. 2025Ev-3DOD: Pushing the Temporal Boundaries of 3D Object Detection with Event Camerascited by 8
  28. 2023PRED: Pre-training via Semantic Rendering on LiDAR Point Cloudscited by 7
  29. 2022Boosting Single-Frame 3D Object Detection by Simulating Multi-Frame Point Cloudscited by 6
  30. 2023Multi-Modal 3D Object Detection by Box Matchingcited by 4
  31. 2025Towards Accurate and Efficient 3D Object Detection for Autonomous Driving: A Mixture of Experts Computing System on Edgecited by 4
  32. 2025Unleashing the Temporal Potential of Stereo Event Cameras for Continuous-Time 3D Object Detectioncited by 4
  33. 2022Cost-Aware Evaluation and Model Scaling for LiDAR-Based 3D Object Detectioncited by 3
  34. 2022VTP: Volumetric Transformer for Multi-view Multi-person 3D Pose Estimationcited by 3
  35. 20233M3D: Multi-view, Multi-path, Multi-representation for 3D Object Detectioncited by 3
  36. 2023Efficient Transformer-based 3D Object Detection with Dynamic Token Haltingcited by 3
  37. 2023MS23D: A 3D Object Detection Method Using Multi-Scale Semantic Feature Points to Construct 3D Feature Layercited by 3
  38. 2023Fuzzy-NMS: Improving 3D Object Detection with Fuzzy Classification in NMScited by 3

50 most cited of 74 All 74 (JSON)