mlsanityData to 14 Oct 2026

PointPainting: Sequential Fusion for 3D Object Detection

arXiv 1911.10150 ↗First posted 22 Nov 2019cs.CV cs.LG eess.IVcited by 449

Not on the map. Why?

Abstract (arXiv)

Camera and lidar are important sensor modalities for robotics in general and self-driving cars in particular. The sensors provide complementary information offering an opportunity for tight sensor-fusion. Surprisingly, lidar-only methods outperform fusion methods on the main benchmark datasets, suggesting a gap in the literature. In this work, we propose PointPainting: a sequential fusion method to fill this gap. PointPainting works by projecting lidar points into the output of an image-only semantic segmentation network and appending the class scores to each point. The appended (painted) point cloud can then be fed to any lidar-only method. Experiments show large improvements on three different state-of-the art methods, Point-RCNN, VoxelNet and PointPillars on the KITTI and nuScenes datasets. The painted version of PointRCNN represents a new state of the art on the KITTI leaderboard for the bird's-eye view detection task. In ablation, we study how the effects of Painting depends on the quality and format of the semantic segmentation output, and demonstrate how latency can be minimized through pipelining.

Key facts. Cited by 449 papers in the map (data through 14 Oct 2026). First posted on arXiv 22 Nov 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 45; 2022: 95; 2023: 124; 2024: 84; 2025: 61; 2026: 40.

449 papers.

Sort:

By area:

  1. 2022BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformerscited by 1,218
  2. 2022BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representationcited by 850
  3. 2022TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Drivingcited by 485
  4. 2022TransFusion: Robust LiDAR-Camera Fusion for 3D Object Detection with Transformerscited by 466
  5. 2022BEVFusion: A Simple and Robust LiDAR-Camera Fusion Frameworkcited by 336
  6. 2021An End-to-End Transformer Model for 3D Object Detectioncited by 329
  7. 2022DAIR-V2X: A Large-Scale Dataset for Vehicle-Infrastructure Cooperative 3D Object Detectioncited by 273
  8. 2022Cross-view Transformers for real-time Map-view Semantic Segmentationcited by 264
  9. 2021PV-RCNN++: Point-Voxel Feature Set Abstraction With Local Vector Representation for 3D Object Detectioncited by 232
  10. 2022DeepFusion: Lidar-Camera Deep Fusion for Multi-Modal 3D Object Detectioncited by 224
  11. 2021One Million Scenes for Autonomous Driving: ONCE Datasetcited by 218
  12. 2022FUTR3D: A Unified Sensor Fusion Framework for 3D Detectioncited by 194
38 more
  1. 2021PandaSet: Advanced Sensor Suite Dataset for Autonomous Drivingcited by 190
  2. 2021Multimodal Virtual Point 3D Detectioncited by 166
  3. 20222DPASS: 2D Priors Assisted Semantic Segmentation on LiDAR Point Cloudscited by 158
  4. 2023V2V4Real: A Real-world Large-scale Dataset for Vehicle-to-Vehicle Cooperative Perceptioncited by 155
  5. 2023DDP: Diffusion Model for Dense Visual Predictioncited by 148
  6. 2023EmbodiedScan: A Holistic Multi-Modal 3D Perception Suite Towards Embodied AIcited by 147
  7. 2023DriveAdapter: Breaking the Coupling Barrier of Perception and Planning in End-to-End Autonomous Drivingcited by 144
  8. 2022Focal Sparse Convolutional Networks for 3D Object Detectioncited by 142
  9. 2022Learning from All Vehiclescited by 135
  10. 20223D Object Detection for Autonomous Driving: A Comprehensive Surveycited by 132
  11. 2022Vision-Centric BEV Perception: A Surveycited by 126
  12. 2025DriveTransformer: Unified Transformer for Scalable End-to-End Autonomous Drivingcited by 126
  13. 2022DeepInteraction: 3D Object Detection via Modality Interactioncited by 125
  14. 2022Delving into the Devils of Bird's-eye-view Perception: A Review, Evaluation and Recipecited by 122
  15. 2021NeSF: Neural Semantic Fields for Generalizable Semantic Segmentation of 3D Scenescited by 119
  16. 2023Benchmarking Robustness of 3D Object Detection to Common Corruptions in Autonomous Drivingcited by 114
  17. 20213D Object Detection for Autonomous Driving: A Surveycited by 106
  18. 2024Towards learning-based planning:The nuPlan benchmark for real-world autonomous drivingcited by 104
  19. 2021Fog Simulation on Real LiDAR Point Clouds for 3D Object Detection in Adverse Weathercited by 97
  20. 2023ReasonNet: End-to-End Driving with Temporal and Global Reasoningcited by 93
  21. 2021FusionPainting: Multimodal Fusion with Adaptive Attention for 3D Object Detectioncited by 92
  22. 2024An Extensible Framework for Open Heterogeneous Collaborative Perceptioncited by 87
  23. 2023SparseFusion: Fusing Multi-Modal Sparse Representations for Multi-Sensor 3D Object Detectioncited by 83
  24. 2022Sparse Fuse Dense: Towards High Quality 3D Detection with Depth Completioncited by 82
  25. 2023CRN: Camera Radar Net for Accurate, Robust, Efficient 3D Perceptioncited by 82
  26. 2023Virtual Sparse Convolution for Multimodal 3D Object Detectioncited by 81
  27. 2023LoGoNet: Towards Accurate 3D Object Detection with Local-to-Global Cross-Modal Fusioncited by 80
  28. 2022Multi-modal Sensor Fusion for Auto Driving Perception: A Surveycited by 76
  29. 2021M3DeTR: Multi-representation, Multi-scale, Mutual-relation 3D Object Detection with Transformerscited by 74
  30. 2023UniTR: A Unified and Efficient Multi-Modal Transformer for Bird's-Eye-View Representationcited by 74
  31. 2022LiDAR Snowfall Simulation for Robust 3D Object Detectioncited by 73
  32. 2023FlatFormer: Flattened Window Attention for Efficient Point Cloud Transformercited by 73
  33. 2023UniPAD: A Universal Pre-training Paradigm for Autonomous Drivingcited by 73
  34. 2022AutoAlign: Pixel-Instance Feature Aggregation for Multi-Modal 3D Object Detectioncited by 70
  35. 2023Cross Modal Transformer: Towards Fast and Robust 3D Object Detectioncited by 70
  36. 2023HM-ViT: Hetero-modal Vehicle-to-Vehicle Cooperative perception with vision transformercited by 70
  37. 2023MSeg3D: Multi-modal 3D Semantic Segmentation for Autonomous Drivingcited by 68
  38. 2022CRAFT: Camera-Radar 3D Object Detection with Spatio-Contextual Fusion Transformercited by 67

50 most cited of 449 All 449 (JSON)