mlsanityData to 14 Oct 2026

PillarNeSt: Embracing Backbone Scaling and Pretraining for Pillar-based 3D Object Detection

arXiv 2311.17770 ↗First posted 29 Nov 2023cs.CV cs.ROcited by 10

Not on the map. Why?

Abstract (arXiv)

This paper shows the effectiveness of 2D backbone scaling and pretraining for pillar-based 3D object detectors. Pillar-based methods mainly employ randomly initialized 2D convolution neural network (ConvNet) for feature extraction and fail to enjoy the benefits from the backbone scaling and pretraining in the image domain. To show the scaling-up capacity in point clouds, we introduce the dense ConvNet pretrained on large-scale image datasets (e.g., ImageNet) as the 2D backbone of pillar-based detectors. The ConvNets are adaptively designed based on the model size according to the specific features of point clouds, such as sparsity and irregularity. Equipped with the pretrained ConvNets, our proposed pillar-based detector, termed PillarNeSt, outperforms the existing 3D object detectors by a large margin on the nuScenes and Argoversev2 datasets. Our code shall be released upon acceptance.

Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Builds on 47 of its 60 references that are in the map. First posted on arXiv 29 Nov 2023.

Builds on

47 of 60 references.

References · 47

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  4. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  5. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  6. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
  7. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,248
  8. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
  9. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,401
  10. 2022A ConvNet for the 2020scited by 3,579
  11. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,375
  12. 2017Rethinking Atrous Convolution for Semantic Image Segmentationcited by 2,161
35 more
  1. 2018PointPillars: Fast Encoders for Object Detection from Point Cloudscited by 1,715
  2. 2017VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detectioncited by 1,323
  3. 2020Center-based 3D Object Detection and Trackingcited by 1,144
  4. 2016Multi-View 3D Object Detection Network for Autonomous Drivingcited by 724
  5. 2023Argoverse 2: Next Generation Datasets for Self-Driving Perception and Forecastingcited by 707
  6. 2017Understanding the Effective Receptive Field in Deep Convolutional Neural Networkscited by 553
  7. 2021RepVGG: Making VGG-style ConvNets Great Againcited by 522
  8. 2022TransFusion: Robust LiDAR-Camera Fusion for 3D Object Detection with Transformerscited by 466
  9. 20203DSSD: Point-based 3D Single Stage Object Detectorcited by 440
  10. 2020Voxel R-CNN: Towards High Performance Voxel-based 3D Object Detectioncited by 410
  11. 2019Class-balanced Grouping and Sampling for Point Cloud 3D Object Detectioncited by 350
  12. 2023VoxelNeXt: Fully Sparse VoxelNet for 3D Object Detection and Trackingcited by 222
  13. 2022Unifying Voxel-based Representation with Transformer for 3D Object Detectioncited by 202
  14. 20203D Object Detection with Pointformercited by 194
  15. 2021Embracing Single Stride 3D Object Detector with Sparse Transformercited by 193
  16. 2020Cylinder3D: An Effective 3D Framework for Driving-scene LiDAR Semantic Segmentationcited by 112
  17. 2020CIA-SSD: Confident IoU-Aware Single-Stage Object Detector From Point Cloudcited by 107
  18. 2022PillarNet: Real-Time and High-Performance Pillar-based 3D Object Detectioncited by 99
  19. 2021AFDetV2: Rethinking the Necessity of the Second Stage for Object Detection from Point Cloudscited by 88
  20. 2019Object as Hotspots: An Anchor-Free 3D Object Detection Approach via Firing of Hotspotscited by 84
  21. 2022Fully Sparse 3D Object Detectioncited by 84
  22. 2020HVNet: Hybrid Voxel Network for LiDAR Based 3D Object Detectioncited by 82
  23. 2023Milestones in Autonomous Driving and Intelligent Vehicles: Survey of Surveyscited by 75
  24. 2022P2P: Tuning Pre-trained Image Models for Point Cloud Analysis with Point-to-Pixel Promptingcited by 74
  25. 2022LargeKernel3D: Scaling up Kernels in 3D Sparse CNNscited by 72
  26. 2020LiDAR-based Online 3D Video Object Detection with Graph-based Message Passing and Spatiotemporal Transformer Attentioncited by 65
  27. 2022Fully Convolutional One-Stage 3D Object Detection on LiDAR Range Imagescited by 56
  28. 2023PillarNeXt: Rethinking Network Designs for 3D Object Detection in LiDAR Point Cloudscited by 54
  29. 2022VISTA: Boosting 3D Object Detection via Dual Cross-VIew SpaTial Attentioncited by 48
  30. 2021HVPR: Hybrid Voxel-Point Representation for Single-stage 3D Object Detectioncited by 47
  31. 2023FastPillars: A Deployment-friendly Pillar-based 3D Detectorcited by 39
  32. 2023LinK: Linear Kernel for LiDAR-based 3D Perceptioncited by 26
  33. 2022DBQ-SSD: Dynamic Ball Query for Efficient 3D Object Detectioncited by 25
  34. 2022BEV-MAE: Bird's Eye View Masked Autoencoders for Point Cloud Pre-training in Autonomous Driving Scenarioscited by 21
  35. 2022A Versatile Multi-View Framework for LiDAR-based 3D Object Detection with Guidance from Panoptic Segmentationcited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 2; 2025: 6; 2026: 2.

10 papers.

  1. 2025MindDrive: A Vision-Language-Action Model for Autonomous Driving via Online Reinforcement Learningcited by 25
  2. 2024Talk2Radar: Bridging Natural Language with 4D mmWave Radar for 3D Referring Expression Comprehensioncited by 18
  3. 2025MetaOcc: Spatio-Temporal Fusion of Surround-View 4D Radar and Camera for 3D Occupancy Prediction with Dual Training Strategiescited by 3
  4. 2025Rethinking Backbone Design for Lightweight 3D Object Detection in LiDARcited by 3
  5. 2025All You Need for Object Detection: From Pixels, Points, and Prompts to Next-Gen Fusion and Multimodal LLMs/VLMs in Autonomous Vehiclescited by 2
  6. 2024Leveraging Anchor-based LiDAR 3D Object Detection via Point Assisted Sample Selectioncited by 1
  7. 2025Pillar-Voxel Fusion Network for 3D Object Detection in Airborne Hyperspectral Point Cloudscited by 1
  8. 2026C2E: Boosting Ego-Only 3D Object Detection via Multi-Teacher Contrastive Knowledge Distillationcited by 1
  9. 2025FALO: Fast and Accurate LiDAR 3D Object Detection on Resource-Constrained Devicescited by 0
  10. 2026Comprehensive Robustness Analysis of LiDAR-based 3D Object Detection in Autonomous Drivingcited by 0