mlsanityData to 7 Oct 2026

MoDAR: Using Motion Forecasting for 3D Object Detection in Point Cloud Sequences

arXiv 2306.03206 ↗First posted 5 Jun 2023cs.CVcited by 13

Not on the map. Why?

Abstract (arXiv)

Occluded and long-range objects are ubiquitous and challenging for 3D object detection. Point cloud sequence data provide unique opportunities to improve such cases, as an occluded or distant object can be observed from different viewpoints or gets better visibility over time. However, the efficiency and effectiveness in encoding long-term sequence data can still be improved. In this work, we propose MoDAR, using motion forecasting outputs as a type of virtual modality, to augment LiDAR point clouds. The MoDAR modality propagates object information from temporal contexts to a target frame, represented as a set of virtual points, one for each object from a waypoint on a forecasted trajectory. A fused point cloud of both raw sensor points and the virtual points can then be fed to any off-the-shelf point-cloud based 3D object detector. Evaluated on the Waymo Open Dataset, our method significantly improves prior art detectors by using motion forecasting from extra-long sequences (e.g. 18 seconds), achieving new state of the arts, while not adding much computation overhead.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Builds on 56 of its 64 references that are in the map. First posted on arXiv 5 Jun 2023.

Builds on

56 of 64 references.

References · 56

  1. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  2. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,436
  3. 2018PointPillars: Fast Encoders for Object Detection from Point Cloudscited by 1,707
  4. 2017VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detectioncited by 1,323
  5. 2019Objects as Pointscited by 1,172
  6. 2020Center-based 3D Object Detection and Trackingcited by 1,143
  7. 2018PointRCNN: 3D Object Proposal Generation and Detection from Point Cloudcited by 939
  8. 2019Deep Hough Voting for 3D Object Detection in Point Cloudscited by 648
  9. 2017Frustum PointNets for 3D Object Detection from RGB-D Datacited by 640
  10. 2021Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Datasetcited by 532
  11. 2019PointPainting: Sequential Fusion for 3D Object Detectioncited by 449
  12. 20203DSSD: Point-based 3D Single Stage Object Detectorcited by 440
38 more
  1. 2018ChauffeurNet: Learning to Drive by Imitating the Best and Synthesizing the Worstcited by 391
  2. 2019PIXOR: Real-time 3D Object Detection from Point Cloudscited by 358
  3. 2020Point-GNN: Graph Neural Network for 3D Object Detection in a Point Cloudcited by 335
  4. 2017Joint 3D Proposal Generation and Object Detection from View Aggregationcited by 325
  5. 2021DenseTNT: End-to-end Trajectory Prediction from Dense Goal Setscited by 292
  6. 2022Wayformer: Motion Forecasting via Simple & Efficient Attention Networkscited by 279
  7. 2021MultiPath++: Efficient Information Fusion and Trajectory Aggregation for Behavior Predictioncited by 246
  8. 2020Fast and Furious: Real Time End-to-End 3D Detection, Tracking and Motion Forecasting with a Single Convolutional Netcited by 228
  9. 2022DeepFusion: Lidar-Camera Deep Fusion for Multi-Modal 3D Object Detectioncited by 224
  10. 2021MP3: A Unified Model to Map, Perceive, Predict and Plancited by 211
  11. 2021Multimodal Motion Prediction with Stacked Transformerscited by 207
  12. 2021Embracing Single Stride 3D Object Detector with Sparse Transformercited by 193
  13. 2021Multimodal Virtual Point 3D Detectioncited by 166
  14. 2021Scene Transformer: A unified architecture for predicting multiple agent trajectoriescited by 165
  15. 2021IntentNet: Learning to Predict Intention from Raw Sensor Datacited by 158
  16. 2021Offboard 3D Object Detection from Point Cloud Sequencescited by 144
  17. 2020PnPNet: End-to-End Perception and Prediction with Tracking in the Loopcited by 143
  18. 2020Pillar-based Object Detection for Autonomous Drivingcited by 137
  19. 2019LaserNet: An Efficient Probabilistic 3D Object Detector for Autonomous Drivingcited by 133
  20. 2021RSN: Range Sparse Net for Efficient, Accurate LiDAR 3D Object Detectioncited by 115
  21. 2021LaneRCNN: Distributed Representations for Graph-Centric Motion Forecastingcited by 112
  22. 2015Deep Sliding Shapes for Amodal 3D Object Detection in RGB-D Imagescited by 111
  23. 2016Vehicle Detection from 3D Lidar Using Fully Convolutional Networkcited by 105
  24. 2020MotionNet: Joint Perception and Motion Prediction for Autonomous Driving Based on Bird's Eye View Mapscited by 100
  25. 2022SWFormer: Sparse Window Transformer for 3D Object Detection in Point Cloudscited by 92
  26. 2020HVNet: Hybrid Voxel Network for LiDAR Based 3D Object Detectioncited by 82
  27. 2022Sparse Fuse Dense: Towards High Quality 3D Detection with Depth Completioncited by 82
  28. 2020Range Conditioned Dilated Convolutions for Scale Invariant 3D Object Detectioncited by 78
  29. 20213D-MAN: 3D Multi-frame Attention Network for Object Detectioncited by 73
  30. 2016Vote3Deep: Fast Object Detection in 3D Point Clouds Using Efficient Convolutional Neural Networkscited by 72
  31. 2022MPPNet: Multi-Frame Feature Intertwining with Proxy Points for 3D Temporal Object Detectioncited by 65
  32. 2020AFDet: Anchor Free One Stage 3D Object Detectioncited by 63
  33. 20163D Fully Convolutional Network for Vehicle Detection in Point Cloudcited by 57
  34. 2018Complex-YOLO: Real-time 3D Object Detection on Point Cloudscited by 56
  35. 20193D Multi-Object Tracking: A Baseline and New Evaluation Metricscited by 47
  36. 2018IPOD: Intensive Point-based Object Detector for Point Cloudcited by 46
  37. 2019StarNet: Targeted Computation for Object Detection in Point Cloudscited by 46
  38. 20214D-Net for Learned Multi-Modal Alignmentcited by 44

50 most cited of 56 All 56 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 4; 2024: 4; 2025: 2; 2026: 3.

13 papers.

Sort:

  1. 2024T4P: Test-Time Training of Trajectory Prediction via Masked Autoencoder and Actor-specific Token Memorycited by 20
  2. 2023PTT: Point-Trajectory Transformer for Efficient Temporal 3D Object Detectioncited by 8
  3. 2023Practical Collaborative Perception: A Framework for Asynchronous and Multi-Agent 3D Object Detectioncited by 6
  4. 2023Frame Fusion with Vehicle Motion Prediction for 3D Object Detectioncited by 3
  5. 2024Future Does Matter: Boosting 3D Object Detection with Temporal Motion Estimation in Point Cloud Sequencescited by 3
  6. 2024Towards Flexible 3D Perception: Object-Centric Occupancy Completion Augments 3D Object Detectioncited by 3
  7. 2025ForeSight: Multi-View Streaming Joint Object Detection and Trajectory Forecastingcited by 3
  8. 2023A survey on deep learning approaches for data integration in autonomous driving systemcited by 1
  9. 2025Is Intermediate Fusion All You Need for UAV-based Collaborative Perception?cited by 1
  10. 2024TrajSSL: Trajectory-Enhanced Semi-Supervised 3D Object Detectioncited by 0
  11. 2026STELLAR: Scaling 3D Perception Large Models for Autonomous Drivingcited by 0
  12. 2026Scene Reconstruction as Mapping Priors for 3D Detectioncited by 0
1 more
  1. 2026CERF: Communication-Efficient and Retraining-Free Collaborative Perceptioncited by 0