# Hidden Gems: 4D Radar Scene Flow Learning Using Cross-Modal Supervision

arXiv: https://arxiv.org/abs/2303.00462 · HTML: https://mlsanity.com/paper/2303.00462/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- Builds on 39 of its 67 references that are in the map.
- First posted on arXiv 1 Mar 2023.

## Builds on (most cited first, 30 of 39)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [PointPillars: Fast Encoders for Object Detection from Point Clouds](https://mlsanity.com/paper/1812.05784/) (2018)
- [VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection](https://mlsanity.com/paper/1711.06396/) (2017)
- [On the Properties of Neural Machine Translation: Encoder-Decoder Approaches](https://mlsanity.com/paper/1409.1259/) (2014)
- [Center-based 3D Object Detection and Tracking](https://mlsanity.com/paper/2006.11275/) (2020)
- [PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume](https://mlsanity.com/paper/1709.02371/) (2017)
- [PointRCNN: 3D Object Proposal Generation and Detection from Point Cloud](https://mlsanity.com/paper/1812.04244/) (2018)
- [A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimation](https://mlsanity.com/paper/1512.02134/) (2015)
- [PointConv: Deep Convolutional Networks on 3D Point Clouds](https://mlsanity.com/paper/1811.07246/) (2018)
- [GeoNet: Unsupervised Learning of Dense Depth, Optical Flow and Camera Pose](https://mlsanity.com/paper/1803.02276/) (2018)
- [FlowNet3D: Learning Scene Flow in 3D Point Clouds](https://mlsanity.com/paper/1806.01411/) (2018)
- [SPLATNet: Sparse Lattice Networks for Point Cloud Processing](https://mlsanity.com/paper/1802.08275/) (2018)
- [DF-Net: Unsupervised Joint Learning of Depth and Flow using Cross-Task Consistency](https://mlsanity.com/paper/1809.01649/) (2018)
- [FLOT: Scene Flow on Point Clouds Guided by Optimal Transport](https://mlsanity.com/paper/2007.11142/) (2020)
- [GNN3DMOT: Graph Neural Network for 3D Multi-Object Tracking with Multi-Feature Learning](https://mlsanity.com/paper/2006.07327/) (2020)
- [HPLFlowNet: Hierarchical Permutohedral Lattice FlowNet for Scene Flow Estimation on Large-scale Point Clouds](https://mlsanity.com/paper/1906.05332/) (2019)
- [Lidar for Autonomous Driving: The principles, challenges, and trends for automotive lidar and perception systems](https://mlsanity.com/paper/2004.08467/) (2020)
- [Just Go with the Flow: Self-Supervised Scene Flow Estimation](https://mlsanity.com/paper/1912.00497/) (2019)
- [Neural Scene Flow Prior](https://mlsanity.com/paper/2111.01253/) (2021)
- [PV-RAFT: Point-Voxel Correlation Fields for Scene Flow Estimation of Point Clouds](https://mlsanity.com/paper/2012.00987/) (2020)
- [FlowStep3D: Model Unrolling for Self-Supervised Scene Flow Estimation](https://mlsanity.com/paper/2011.10147/) (2020)
- [Occlusions, Motion and Depth Boundaries with a Generic Network for Disparity, Optical Flow or Scene Flow Estimation](https://mlsanity.com/paper/1808.01838/) (2018)
- [Self-Supervised Monocular Scene Flow Estimation](https://mlsanity.com/paper/2004.04143/) (2020)
- [Weakly Supervised Learning of Rigid 3D Scene Flow](https://mlsanity.com/paper/2102.08945/) (2021)
- [FlowNet3D++: Geometric Losses For Deep Scene Flow Estimation](https://mlsanity.com/paper/1912.01438/) (2019)
- [Scalable Scene Flow from Point Clouds in the Real World](https://mlsanity.com/paper/2103.01306/) (2021)
- [Scene Flow from Point Clouds with or without Learning](https://mlsanity.com/paper/2011.00320/) (2020)

## Built on by (most cited first, 30 of 36)

- [SMURF: Spatial Multi-Representation Fusion for 3D Object Detection with 4D Imaging Radar](https://mlsanity.com/paper/2307.10784/) (2023)
- [Unleashing HyDRa: Hybrid Fusion, Depth Consistency and Radar for Unified 3D Perception](https://mlsanity.com/paper/2403.07746/) (2024)
- [RadarOcc: Robust 3D Occupancy Prediction with 4D Imaging Radar](https://mlsanity.com/paper/2405.14014/) (2024)
- [Exploring Radar Data Representations in Autonomous Driving: A Comprehensive Review](https://mlsanity.com/paper/2312.04861/) (2023)
- [Doracamom: Joint 3D Detection and Occupancy Prediction with Multi-view 4D Radars and Cameras for Omnidirectional Perception](https://mlsanity.com/paper/2501.15394/) (2025)
- [RaTrack: Moving Object Detection and Tracking with 4D Radar Point Cloud](https://mlsanity.com/paper/2309.09737/) (2023)
- [milliFlow: Scene Flow Estimation on mmWave Radar Point Cloud for Human Motion Sensing](https://mlsanity.com/paper/2306.17010/) (2023)
- [DART: Implicit Doppler Tomography for Radar Novel View Synthesis](https://mlsanity.com/paper/2403.03896/) (2024)
- [Radar Fields: Frequency-Space Neural Scene Representations for FMCW Radar](https://mlsanity.com/paper/2405.04662/) (2024)
- [Towards Dense and Accurate Radar Perception Via Efficient Cross-Modal Diffusion Model](https://mlsanity.com/paper/2403.08460/) (2024)
- [Bring Event into RGB and LiDAR: Hierarchical Visual-Motion Fusion for Scene Flow](https://mlsanity.com/paper/2403.07432/) (2024)
- [4DRVO-Net: Deep 4D Radar-Visual Odometry Using Multi-Modal and Multi-Scale Adaptive Fusion](https://mlsanity.com/paper/2308.06573/) (2023)
- [Radars for Autonomous Driving: A Review of Deep Learning Methods and Challenges](https://mlsanity.com/paper/2306.09304/) (2023)
- [Radar Instance Transformer: Reliable Moving Instance Segmentation in Sparse Radar Point Clouds](https://mlsanity.com/paper/2309.16435/) (2023)
- [Bootstrapping Autonomous Driving Radars with Self-Supervised Learning](https://mlsanity.com/paper/2312.04519/) (2023)
- [Radar and Camera Fusion for Object Detection and Tracking: A Comprehensive Survey](https://mlsanity.com/paper/2410.19872/) (2024)
- [TacoDepth: Towards Efficient Radar-Camera Depth Estimation with One-stage Fusion](https://mlsanity.com/paper/2504.11773/) (2025)
- [M4Human: A Large-Scale Multimodal mmWave Radar Benchmark for Human Mesh Reconstruction](https://mlsanity.com/paper/2512.12378/) (2025)
- [Self-Supervised Bird's Eye View Motion Prediction with Cross-Modality Signals](https://mlsanity.com/paper/2401.11499/) (2024)
- [SIRA: Scalable Inter-frame Relation and Association for Radar Perception](https://mlsanity.com/paper/2411.02220/) (2024)
- [DoGFlow: Self-Supervised LiDAR Scene Flow via Cross-Modal Doppler Guidance](https://mlsanity.com/paper/2508.18506/) (2025)
- [Instantaneous Perception of Moving Objects in 3D](https://mlsanity.com/paper/2405.02781/) (2024)
- [LEROjD: Lidar Extended Radar-Only Object Detection](https://mlsanity.com/paper/2409.05564/) (2024)
- [TARS: Traffic-Aware Radar Scene Flow Estimation](https://mlsanity.com/paper/2503.10210/) (2025)
- [RaLiFlow: Scene Flow Estimation with 4D Radar and LiDAR Point Clouds](https://mlsanity.com/paper/2512.10376/) (2025)
- [HyperDet: 3D Object Detection with Hyper 4D Radar Point Clouds](https://mlsanity.com/paper/2602.11554/) (2026)
- [RadarMP: Motion Perception for 4D mmWave Radar in Autonomous Driving](https://mlsanity.com/paper/2511.12117/) (2025)
- [RISE: Single Static Radar-based Indoor Scene Understanding](https://mlsanity.com/paper/2511.14019/) (2025)
- [RLM: A Vision-Language Model Approach for Radar Scene Understanding](https://mlsanity.com/paper/2511.21105/) (2025)
- [RadarGen: Automotive Radar Point Cloud Generation from Cameras](https://mlsanity.com/paper/2512.17897/) (2025)

## Abstract (arXiv)

This work proposes a novel approach to 4D radar-based scene flow estimation via cross-modal learning. Our approach is motivated by the co-located sensing redundancy in modern autonomous vehicles. Such redundancy implicitly provides various forms of supervision cues to the radar scene flow estimation. Specifically, we introduce a multi-task model architecture for the identified cross-modal learning problem and propose loss functions to opportunistically engage scene flow estimation using multiple cross-modal constraints for effective model training. Extensive experiments show the state-of-the-art performance of our method and demonstrate the effectiveness of cross-modal supervised learning to infer more accurate 4D radar scene flow. We also show its usefulness to two subtasks - motion segmentation and ego-motion estimation. Our source code will be available on https://github.com/Toytiny/CMFlow.