mlsanityData to 7 Oct 2026

4D-Net for Learned Multi-Modal Alignment

arXiv 2109.01066 ↗First posted 2 Sep 2021cs.CVcited by 44

Not on the map. Why?

Abstract (arXiv)

We present 4D-Net, a 3D object detection approach, which utilizes 3D Point Cloud and RGB sensing information, both in time. We are able to incorporate the 4D information by performing a novel dynamic connection learning across various feature representations and levels of abstraction, as well as by observing geometric constraints. Our approach outperforms the state-of-the-art and strong baselines on the Waymo Open Dataset. 4D-Net is better able to use motion cues and dense image information to detect distant objects more successfully.

Key facts. Cited by 44 papers in the map (data through 7 Oct 2026). Builds on 44 of its 61 references that are in the map. First posted on arXiv 2 Sep 2021.

Builds on

44 of 61 references.

References · 44

  1. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
  2. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  3. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,427
  4. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,436
  5. 2013Rich feature hierarchies for accurate object detection and semantic segmentationcited by 2,160
  6. 2018DARTS: Differentiable Architecture Searchcited by 1,782
  7. 2018PointPillars: Fast Encoders for Object Detection from Point Cloudscited by 1,707
  8. 2015Spatial Transformer Networkscited by 1,423
  9. 2017VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detectioncited by 1,323
  10. 20194D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networkscited by 1,201
  11. 2017A Closer Look at Spatiotemporal Convolutions for Action Recognitioncited by 1,139
  12. 2018PointRCNN: 3D Object Proposal Generation and Detection from Point Cloudcited by 939
32 more
  1. 2019PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detectioncited by 890
  2. 2016Multi-View 3D Object Detection Network for Autonomous Drivingcited by 721
  3. 2017Frustum PointNets for 3D Object Detection from RGB-D Datacited by 640
  4. 2019PointPainting: Sequential Fusion for 3D Object Detectioncited by 449
  5. 2019PIXOR: Real-time 3D Object Detection from Point Cloudscited by 358
  6. 2017Joint 3D Proposal Generation and Object Detection from View Aggregationcited by 325
  7. 2020Deep Continuous Fusion for Multi-Sensor 3D Object Detectioncited by 263
  8. 2020Multi-Task Multi-Sensor Fusion for 3D Object Detectioncited by 203
  9. 2017Learning Spatio-Temporal Features with 3D Residual Networks for Action Recognitioncited by 178
  10. 2019End-to-End Multi-View Fusion for 3D Object Detection in LiDAR Point Cloudscited by 171
  11. 2017PointFusion: Deep Sensor Fusion for 3D Bounding Box Estimationcited by 170
  12. 2018SPLATNet: Sparse Lattice Networks for Point Cloud Processingcited by 158
  13. 2015End-to-end people detection in crowded scenescited by 140
  14. 2020Pillar-based Object Detection for Autonomous Drivingcited by 137
  15. 2019LaserNet: An Efficient Probabilistic 3D Object Detector for Autonomous Drivingcited by 133
  16. 2014Learning Rich Features from RGB-D Images for Object Detection and Segmentationcited by 124
  17. 2020Kinematic 3D Object Detection in Monocular Videocited by 116
  18. 2016Vehicle Detection from 3D Lidar Using Fully Convolutional Networkcited by 105
  19. 2016A Unified Multi-scale Deep Convolutional Neural Network for Fast Object Detectioncited by 83
  20. 2020Range Conditioned Dilated Convolutions for Scale Invariant 3D Object Detectioncited by 78
  21. 2020HDNET: Exploiting HD Maps for 3D Object Detectioncited by 78
  22. 2016Vote3Deep: Fast Object Detection in 3D Point Clouds Using Efficient Convolutional Neural Networkscited by 72
  23. 2019What You See is What You Get: Exploiting Visibility for 3D Object Detectioncited by 69
  24. 2020Detection in Crowded Scenes: One Proposal, Multiple Predictionscited by 59
  25. 20163D Fully Convolutional Network for Vehicle Detection in Point Cloudcited by 57
  26. 2019AssembleNet: Searching for Multi-Stream Neural Connectivity in Video Architecturescited by 46
  27. 2019StarNet: Targeted Computation for Object Detection in Point Cloudscited by 46
  28. 2020An LSTM Approach to Temporal 3D Object Detection in LiDAR Point Cloudscited by 45
  29. 2020AssembleNet++: Assembling Modality Representations via Attention Connectionscited by 28
  30. 2017Illuminating Pedestrians via Simultaneous Detection & Segmentationcited by 26
  31. 2020STINet: Spatio-Temporal-Interactive Network for Pedestrian Detection and Trajectory Predictioncited by 22
  32. 2014Ten Years of Pedestrian Detection, What Have We Learned?cited by 13

Built on by

Cited by year

Citing papers per year: 2021: 2; 2022: 18; 2023: 9; 2024: 10; 2025: 5.

44 papers.

Sort:

By area:

  1. 2022HOI4D: A 4D Egocentric Dataset for Category-Level Human-Object Interactioncited by 321
  2. 2022DeepFusion: Lidar-Camera Deep Fusion for Multi-Modal 3D Object Detectioncited by 224
  3. 20223D Object Detection for Autonomous Driving: A Comprehensive Surveycited by 132
  4. 2022Vision-Centric BEV Perception: A Surveycited by 126
  5. 2022DeepInteraction: 3D Object Detection via Modality Interactioncited by 125
  6. 2023FocalFormer3D : Focusing on Hard Instance for 3D Object Detectioncited by 58
  7. 2024IS-Fusion: Instance-Scene Collaborative Fusion for Multimodal 3D Object Detectioncited by 57
  8. 2022Beyond 3D Siamese Tracking: A Motion-Centric Paradigm for 3D Single Object Tracking in Point Cloudscited by 55
  9. 2021EPNet++: Cascade Bi-directional Fusion for Multi-Modal 3D Object Detectioncited by 49
  10. 2022Dynamic 3D Scene Analysis by Point Cloud Accumulationcited by 47
  11. 2023BEVFusion4D: Learning LiDAR-Camera Fusion Under Bird's-Eye-View via Cross-Modality Guidance and Temporal Aggregationcited by 31
  12. 2022CramNet: Camera-Radar Fusion with Ray-Constrained Cross-Attention for Robust 3D Object Detectioncited by 30
32 more
  1. 2022Weakly Supervised 3D Multi-person Pose Estimation for Large-scale Scenes based on Monocular Camera and Single LiDARcited by 23
  2. 20224D Unsupervised Object Discoverycited by 18
  3. 2024DeepInteraction++: Multi-Modality Interaction for Autonomous Drivingcited by 15
  4. 2022INT: Towards Infinite-frames 3D Detection with An Efficient Frameworkcited by 14
  5. 2023Transformer-Based Sensor Fusion for Autonomous Driving: A Surveycited by 14
  6. 2023An Effective Motion-Centric Paradigm for 3D Single Object Tracking in Point Cloudscited by 14
  7. 2022Learning Viewpoint-Agnostic Visual Representations by Recovering Tokens in 3D Spacecited by 13
  8. 2023MoDAR: Using Motion Forecasting for 3D Object Detection in Point Cloud Sequencescited by 13
  9. 2022NVRadarNet: Real-Time Radar Obstacle and Free Space Detection for Autonomous Drivingcited by 12
  10. 2022A Survey of Robust 3D Object Detection Methods in Point Cloudscited by 10
  11. 2023Lift-Attend-Splat: Bird's-eye-view camera-lidar fusion using transformerscited by 8
  12. 2025Ev-3DOD: Pushing the Temporal Boundaries of 3D Object Detection with Event Camerascited by 8
  13. 2022PAI3D: Painting Adaptive Instance-Prior for 3D Object Detectioncited by 6
  14. 2022Sequential Point Clouds: A Surveycited by 5
  15. 2024UFO: Uncertainty-aware LiDAR-image Fusion for Off-road Semantic Terrain Map Estimationcited by 5
  16. 2024MARS: Multimodal Active Robotic Sensing for Articulated Characterizationcited by 5
  17. 2022Time-to-Label: Temporal Consistency for Self-Supervised Monocular 3D Object Detectioncited by 3
  18. 2023Frame Fusion with Vehicle Motion Prediction for 3D Object Detectioncited by 3
  19. 2025Unleashing the Temporal Potential of Stereo Event Cameras for Continuous-Time 3D Object Detectioncited by 3
  20. 2024Sports Analysis and VR Viewing System Based on Player Tracking and Pose Estimation with Multimodal and Multiview Sensorscited by 2
  21. 2021Dynamic programming with incomplete information to overcome navigational uncertainty in a nautical environmentcited by 1
  22. 2023MaskedFusion360: Reconstruct LiDAR Data by Querying Camera Featurescited by 1
  23. 2023LiCamPose: Combining Multi-View LiDAR and RGB Cameras for Robust Single-timestamp 3D Human Pose Estimationcited by 1
  24. 2024AYDIV: Adaptable Yielding 3D Object Detection via Integrated Contextual Vision Transformercited by 1
  25. 2024AdaptiveFusion: Adaptive Multi-Modal Multi-View Fusion for 3D Human Body Reconstructioncited by 1
  26. 2025Semantic-Supervised Spatial-Temporal Fusion for LiDAR-based 3D Object Detectioncited by 1
  27. 2025M^3Detection: Multi-Frame Multi-Level Feature Fusion for Multi-Modal 3D Object Detection with Camera and 4D Imaging Radarcited by 1
  28. 2022ImLiDAR: Cross-Sensor Dynamic Message Propagation Network for 3D Object Detectioncited by 0
  29. 2024Interactive Test-Time Adaptation with Reliable Spatial-Temporal Voxels for Multi-Modal Segmentationcited by 0
  30. 2024Quantum Inverse Contextual Vision Transformers (Q-ICVT): A New Frontier in 3D Object Detection for AVscited by 0
  31. 2024Adversarial Attacks on LiDAR-Based Tracking Across Road Users: Robustness Evaluation and Target-Aware Black-Box Methodcited by 0
  32. 2025End-to-End 3-D Spatiotemporal Perception with Multimodal Fusion and V2X Collaborationcited by 0