mlsanityData to 14 Oct 2026

A survey on deep learning approaches for data integration in autonomous driving system

arXiv 2306.11740 ↗First posted 17 Jun 2023cs.RO cs.CV cs.LGcited by 1

Not on the map. Why?

Abstract (arXiv)

The perception module of self-driving vehicles relies on a multi-sensor system to understand its environment. Recent advancements in deep learning have led to the rapid development of approaches that integrate multi-sensory measurements to enhance perception capabilities. This paper surveys the latest deep learning integration techniques applied to the perception module in autonomous driving systems, categorizing integration approaches based on "what, how, and when to integrate". A new taxonomy of integration is proposed, based on three dimensions: multi-view, multi-modality, and multi-frame. The integration operations and their pros and cons are summarized, providing new insights into the properties of an "ideal" data integration approach that can alleviate the limitations of existing methods. After reviewing hundreds of relevant papers, this survey concludes with a discussion of the key features of an optimal data integration approach.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 151 of its 279 references that are in the map. First posted on arXiv 17 Jun 2023.

Builds on

151 of 279 references.

References · 151

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,248
  3. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,401
  4. 2021Is Space-Time Attention All You Need for Video Understanding?cited by 1,620
  5. 2017VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detectioncited by 1,323
  6. 2022BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformerscited by 1,218
  7. 2019KPConv: Flexible and Deformable Convolution for Point Cloudscited by 1,205
  8. 2020Center-based 3D Object Detection and Trackingcited by 1,144
  9. 2019PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detectioncited by 891
  10. 2022BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representationcited by 850
  11. 2018MVSNet: Depth Inference for Unstructured Multi-view Stereocited by 788
  12. 2017Unsupervised Learning of Depth and Ego-Motion from Videocited by 782
38 more
  1. 2018PointCNN: Convolution On X-Transformed Pointscited by 764
  2. 2021DROID-SLAM: Deep Visual SLAM for Monocular, Stereo, and RGB-D Camerascited by 687
  3. 2021DETR3D: 3D Object Detection from Multi-view Images via 3D-to-2D Queriescited by 658
  4. 2017Frustum PointNets for 3D Object Detection from RGB-D Datacited by 640
  5. 2020X3D: Expanding Architectures for Efficient Video Recognitioncited by 637
  6. 2022PETR: Position Embedding Transformation for Multi-View 3D Object Detectioncited by 552
  7. 2017Can Spatiotemporal 3D CNNs Retrace the History of 2D CNNs and ImageNet?cited by 551
  8. 2021TrackFormer: Multi-Object Tracking with Transformerscited by 518
  9. 2020Tracking Objects as Pointscited by 483
  10. 2022TransFusion: Robust LiDAR-Camera Fusion for 3D Object Detection with Transformerscited by 466
  11. 2019PointPainting: Sequential Fusion for 3D Object Detectioncited by 449
  12. 2019Cascade Cost Volume for High-Resolution Multi-View Stereo and Stereo Matchingcited by 417
  13. 2021MOTR: End-to-End Multiple-Object Tracking with Transformercited by 381
  14. 2020TransTrack: Multiple Object Tracking with Transformercited by 371
  15. 2022PETRv2: A Unified Framework for 3D Perception from Multi-Camera Imagescited by 352
  16. 2022Observation-Centric SORT: Rethinking SORT for Robust Multi-Object Trackingcited by 346
  17. 2019Point-Voxel CNN for Efficient 3D Deep Learningcited by 343
  18. 2018GeoNet: Unsupervised Learning of Dense Depth, Optical Flow and Camera Posecited by 342
  19. 2022BEVDet4D: Exploit Temporal Cues in Multi-camera 3D Object Detectioncited by 341
  20. 2019Deep Multi-modal Object Detection and Semantic Segmentation for Autonomous Driving: Datasets, Methods, and Challengescited by 314
  21. 2019Med3D: Transfer Learning for 3D Medical Image Analysiscited by 288
  22. 2022Cross-view Transformers for real-time Map-view Semantic Segmentationcited by 264
  23. 2020Deep Continuous Fusion for Multi-Sensor 3D Object Detectioncited by 263
  24. 2021NeuralRecon: Real-Time Coherent 3D Reconstruction from Monocular Videocited by 241
  25. 2021FIERY: Future Instance Prediction in Bird's-Eye View from Surround Monocular Camerascited by 239
  26. 2022DeepFusion: Lidar-Camera Deep Fusion for Multi-Modal 3D Object Detectioncited by 224
  27. 2019MOTS: Multi-Object Tracking and Segmentationcited by 223
  28. 2020Atlas: End-to-End 3D Scene Reconstruction from Posed Imagescited by 206
  29. 2020Multi-Task Multi-Sensor Fusion for 3D Object Detectioncited by 203
  30. 2022FUTR3D: A Unified Sensor Fusion Framework for 3D Detectioncited by 194
  31. 2022BEVerse: Unified Perception and Prediction in Birds-Eye-View for Vision-Centric Autonomous Drivingcited by 191
  32. 2021The Temporal Opportunist: Self-Supervised Multi-Frame Monocular Depthcited by 190
  33. 2022UniFormer: Unified Transformer for Efficient Spatiotemporal Representation Learningcited by 184
  34. 2017Computer Vision for Autonomous Vehicles: Problems, Datasets and State of the Artcited by 182
  35. 2017DeepVO: Towards End-to-End Visual Odometry with Deep Recurrent Convolutional Neural Networkscited by 175
  36. 2019Cross-view Semantic Segmentation for Sensing Surroundingscited by 174
  37. 2019End-to-End Multi-View Fusion for 3D Object Detection in LiDAR Point Cloudscited by 171
  38. 2017PointFusion: Deep Sensor Fusion for 3D Bounding Box Estimationcited by 170

50 most cited of 151 All 151 (JSON)

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025A Novel Multi-layer Task-centric and Data Quality Framework for Autonomous Drivingcited by 2