mlsanityData to 14 Oct 2026

PVNAS: 3D Neural Architecture Search with Point-Voxel Convolution

arXiv 2204.11797 ↗First posted 25 Apr 2022cs.CV cs.LGcited by 16

Not on the map. Why?

Abstract (arXiv)

3D neural networks are widely used in real-world applications (e.g., AR/VR headsets, self-driving cars). They are required to be fast and accurate; however, limited hardware resources on edge devices make these requirements rather challenging. Previous work processes 3D data using either voxel-based or point-based neural networks, but both types of 3D models are not hardware-efficient due to the large memory footprint and random memory access. In this paper, we study 3D deep learning from the efficiency perspective. We first systematically analyze the bottlenecks of previous 3D methods. We then combine the best from point-based and voxel-based models together and propose a novel hardware-efficient 3D primitive, Point-Voxel Convolution (PVConv). We further enhance this primitive with the sparse convolution to make it more effective in processing large (outdoor) scenes. Based on our designed 3D primitive, we introduce 3D Neural Architecture Search (3D-NAS) to explore the best 3D network architecture given a resource constraint. We evaluate our proposed method on six representative benchmark datasets, achieving state-of-the-art performance with 1.8-23.7x measured speedup. Furthermore, our method has been deployed to the autonomous racing vehicle of MIT Driverless, achieving larger detection range, higher accuracy and lower latency.

Key facts. Cited by 16 papers in the map (data through 14 Oct 2026). Builds on 78 of its 93 references that are in the map. First posted on arXiv 25 Apr 2022.

Builds on

78 of 93 references.

References · 78

  1. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 7,005
  2. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
  3. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,248
  4. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,795
  5. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
  6. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,401
  7. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,845
  8. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,897
  9. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,329
  10. 2018Dynamic Graph CNN for Learning on Point Cloudscited by 2,223
  11. 2015Learning both Weights and Connections for Efficient Neural Networkscited by 2,031
  12. 2019Searching for MobileNetV3cited by 1,994
38 more
  1. 2018DARTS: Differentiable Architecture Searchcited by 1,798
  2. 2016Neural Architecture Search with Reinforcement Learningcited by 1,499
  3. 20163D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotationcited by 1,476
  4. 2017VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detectioncited by 1,323
  5. 2019Energy and Policy Considerations for Deep Learning in NLPcited by 1,254
  6. 20194D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networkscited by 1,208
  7. 2019KPConv: Flexible and Deformable Convolution for Point Cloudscited by 1,205
  8. 2017ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devicescited by 1,200
  9. 2018ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Designcited by 1,136
  10. 2017Learning Transferable Architectures for Scalable Image Recognitioncited by 1,101
  11. 2016SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model sizecited by 1,060
  12. 2018MnasNet: Platform-Aware Neural Architecture Search for Mobilecited by 943
  13. 2018PointRCNN: 3D Object Proposal Generation and Detection from Point Cloudcited by 941
  14. 2019Once-for-All: Train One Network and Specialize it for Efficient Deploymentcited by 787
  15. 2018PointCNN: Convolution On X-Transformed Pointscited by 764
  16. 20173D Semantic Segmentation with Submanifold Sparse Convolutional Networkscited by 748
  17. 2018ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardwarecited by 673
  18. 2018PointConv: Deep Convolutional Networks on 3D Point Cloudscited by 657
  19. 2017Frustum PointNets for 3D Object Detection from RGB-D Datacited by 640
  20. 2019RandLA-Net: Efficient Semantic Segmentation of Large-Scale Point Cloudscited by 585
  21. 2019DeepGCNs: Can GCNs Go as Deep as CNNs?cited by 527
  22. 2018PartNet: A Large-scale Benchmark for Fine-grained and Hierarchical Part-level 3D Object Understandingcited by 526
  23. 2017Progressive Neural Architecture Searchcited by 487
  24. 2018FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Searchcited by 485
  25. 2018HAQ: Hardware-Aware Automated Quantization with Mixed Precisioncited by 443
  26. 2019Single Path One-Shot Neural Architecture Search with Uniform Samplingcited by 414
  27. 2017Joint 2D-3D-Semantic Data for Indoor Scene Understandingcited by 406
  28. 2017Large-scale Point Cloud Semantic Segmentation with Superpoint Graphscited by 385
  29. 2018AMC: AutoML for Model Compression and Acceleration on Mobile Devicescited by 378
  30. 2016OctNet: Learning Deep 3D Representations at High Resolutionscited by 342
  31. 2020PolarNet: An Improved Grid Representation for Online LiDAR Point Clouds Semantic Segmentationcited by 275
  32. 2018SpiderCNN: Deep Learning on Point Sets with Parameterized Convolutional Filterscited by 261
  33. 2016Volumetric and Multi-View CNNs for Object Classification on 3D Datacited by 240
  34. 2019MetaPruning: Meta Learning for Automatic Neural Network Channel Pruningcited by 227
  35. 2020SalsaNext: Fast, Uncertainty-aware Semantic Segmentation of LiDAR Point Clouds for Autonomous Drivingcited by 226
  36. 2017Incremental Network Quantization: Towards Lossless CNNs with Low-Precision Weightscited by 217
  37. 2018SO-Net: Self-Organizing Network for Point Cloud Analysiscited by 210
  38. 2017SEGCloud: Semantic Segmentation of 3D Point Cloudscited by 179

50 most cited of 78 All 78 (JSON)

Built on by

Cited by year

Citing papers per year: 2022: 5; 2023: 8; 2024: 1; 2025: 2.

16 papers.

Sort:

By area:

  1. 2022BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representationcited by 850
  2. 2022Learning to Generate Realistic LiDAR Point Cloudscited by 93
  3. 2023FlatFormer: Flattened Window Attention for Efficient Point Cloud Transformercited by 73
  4. 2023SparseViT: Revisiting Activation Sparsity for Efficient High-Resolution Vision Transformercited by 60
  5. 2023PillarNeXt: Rethinking Network Designs for 3D Object Detection in LiDAR Point Cloudscited by 54
  6. 2023ProxyFormer: Proxy Alignment Assisted Point Cloud Completion with Missing Part Sensitive Transformercited by 43
  7. 2023TorchSparse++: Efficient Training and Inference Framework for Sparse Convolution on GPUscited by 36
  8. 2022Analyzing Deep Learning Representations of Point Clouds for Real-Time In-Vehicle LiDAR Perceptioncited by 8
  9. 2023CurveCloudNet: Processing Point Clouds with 1D Structurecited by 5
  10. 2024Efficient Deep Learning Infrastructures for Embedded Computing Systems: A Comprehensive Survey and Future Envisioncited by 5
  11. 2022Extensible Proxy for Efficient NAScited by 1
  12. 2022Point-Voxel Adaptive Feature Abstraction for Robust Point Cloud Classificationcited by 1
4 more
  1. 2025Querying Autonomous Vehicle Point Clouds: Enhanced by 3D Object Counting with CounterNetcited by 1
  2. 2023Comparative study of subset selection methods for rapid prototyping of 3D object detection algorithmscited by 0
  3. 2023HYVE: Hybrid Vertex Encoder for Neural Distance Fieldscited by 0
  4. 2025LiSTAR: Ray-Centric World Models for 4D LiDAR Sequences in Autonomous Drivingcited by 0