mlsanityData to 7 Oct 2026

Self-Supervised Bird's Eye View Motion Prediction with Cross-Modality Signals

arXiv 2401.11499 ↗First posted 21 Jan 2024cs.CVcited by 5

Not on the map. Why?

Abstract (arXiv)

Learning the dense bird's eye view (BEV) motion flow in a self-supervised manner is an emerging research for robotics and autonomous driving. Current self-supervised methods mainly rely on point correspondences between point clouds, which may introduce the problems of fake flow and inconsistency, hindering the model's ability to learn accurate and realistic motion. In this paper, we introduce a novel cross-modality self-supervised training framework that effectively addresses these issues by leveraging multi-modality data to obtain supervision signals. We design three innovative supervision signals to preserve the inherent properties of scene motion, including the masked Chamfer distance loss, the piecewise rigidity loss, and the temporal consistency loss. Through extensive experiments, we demonstrate that our proposed self-supervised framework outperforms all previous self-supervision methods for the motion prediction task.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 38 of its 46 references that are in the map. First posted on arXiv 21 Jan 2024.

Builds on

38 of 46 references.

References · 38

  1. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  2. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,385
  3. 2018PointPillars: Fast Encoders for Object Detection from Point Cloudscited by 1,707
  4. 2017PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volumecited by 966
  5. 2018PointRCNN: 3D Object Proposal Generation and Detection from Point Cloudcited by 939
  6. 2023Argoverse 2: Next Generation Datasets for Self-Driving Perception and Forecastingcited by 701
  7. 2019PointPainting: Sequential Fusion for 3D Object Detectioncited by 449
  8. 2022BEVFusion: A Simple and Robust LiDAR-Camera Fusion Frameworkcited by 334
  9. 2018FlowNet3D: Learning Scene Flow in 3D Point Cloudscited by 241
  10. 2021FIERY: Future Instance Prediction in Bird's-Eye View from Surround Monocular Camerascited by 239
  11. 2020Fast and Furious: Real Time End-to-End 3D Detection, Tracking and Motion Forecasting with a Single Convolutional Netcited by 228
  12. 2022BEVerse: Unified Perception and Prediction in Birds-Eye-View for Vision-Centric Autonomous Drivingcited by 191
26 more
  1. 2020FLOT: Scene Flow on Point Clouds Guided by Optimal Transportcited by 129
  2. 2019HPLFlowNet: Hierarchical Permutohedral Lattice FlowNet for Scene Flow Estimation on Large-scale Point Cloudscited by 108
  3. 2019Spatially-Aware Graph Neural Networks for Relational Behavior Forecasting from Sensor Datacited by 104
  4. 2020MotionNet: Joint Perception and Motion Prediction for Autonomous Driving Based on Bird's Eye View Mapscited by 100
  5. 2019Just Go with the Flow: Self-Supervised Scene Flow Estimationcited by 98
  6. 2021Neural Scene Flow Priorcited by 91
  7. 2023Point Cloud Forecasting as a Proxy for 4D Occupancy Forecastingcited by 76
  8. 2019Identifying Unknown Instances for Autonomous Drivingcited by 73
  9. 2020FlowStep3D: Model Unrolling for Self-Supervised Scene Flow Estimationcited by 71
  10. 20203D Point Cloud Processing and Learning for Autonomous Drivingcited by 68
  11. 2021Weakly Supervised Learning of Rigid 3D Scene Flowcited by 60
  12. 2023Asynchrony-Robust Collaborative Perception via Bird's Eye View Flowcited by 58
  13. 2021CamLiFlow: Bidirectional Camera-LiDAR Fusion for Joint Optical Flow and Scene Flow Estimationcited by 56
  14. 2021Scalable Scene Flow from Point Clouds in the Real Worldcited by 49
  15. 2020Scene Flow from Point Clouds with or without Learningcited by 44
  16. 2021Self-Supervised Pillar Motion Learning for Autonomous Drivingcited by 44
  17. 2020Self-Supervised Learning of Non-Rigid Residual Flow and Ego-Motioncited by 41
  18. 2019PointPWC-Net: A Coarse-to-Fine Network for Supervised and Self-Supervised Scene Flow Estimation on 3D Point Cloudscited by 38
  19. 2022Bi-PointFlowNet: Bidirectional Learning for Point Cloud Based Scene Flow Estimationcited by 38
  20. 2021HCRF-Flow: Scene Flow from Point Clouds with Continuous High-order CRFs and Position-aware Flow Embeddingcited by 36
  21. 2023Hidden Gems: 4D Radar Scene Flow Learning Using Cross-Modal Supervisioncited by 36
  22. 2018Long-Term Occupancy Grid Prediction Using Recurrent Neural Networkscited by 29
  23. 2023Self-Supervised 3D Scene Flow Estimation Guided by Superpointscited by 24
  24. 2021Deep Multi-Task Learning for Joint Localization, Perception, and Predictioncited by 22
  25. 2020PillarFlow: End-to-end Birds-eye-view Flow Estimation for Autonomous Drivingcited by 16
  26. 2022Learning Scene Flow in 3D Point Clouds with Noisy Pseudo Labelscited by 3

Built on by

Cited by year

Citing papers per year: 2024: 3; 2025: 2.

5 papers.

  1. 2024Joint Perception and Prediction for Autonomous Driving: A Surveycited by 7
  2. 2025LEGO-Motion: Learning-Enhanced Grids with Occupancy Instance Modeling for Class-Agnostic Motion Predictioncited by 6
  3. 2024FedRSU: Federated Learning for Scene Flow Estimation on Roadside Unitscited by 2
  4. 2025Weakly and Self-Supervised Class-Agnostic Motion Prediction for Autonomous Drivingcited by 2
  5. 2024Long-Term 3D Point Tracking By Cost Volume Fusioncited by 0