# DeepLiDARFlow: A Deep Learning Architecture For Scene Flow Estimation Using Monocular Camera and Sparse LiDAR

arXiv: https://arxiv.org/abs/2008.08136 · HTML: https://mlsanity.com/paper/2008.08136/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Aug 2020.

## Built on by (most cited first, 24 of 24)

- [Neural Scene Flow Prior](https://mlsanity.com/paper/2111.01253/) (2021)
- [CamLiFlow: Bidirectional Camera-LiDAR Fusion for Joint Optical Flow and Scene Flow Estimation](https://mlsanity.com/paper/2111.10502/) (2021)
- [Self-Supervised Pillar Motion Learning for Autonomous Driving](https://mlsanity.com/paper/2104.08683/) (2021)
- [Fast Neural Scene Flow](https://mlsanity.com/paper/2304.09121/) (2023)
- [RPEFlow: Multimodal Fusion of RGB-PointCloud-Event for Joint Optical Flow and Scene Flow Estimation](https://mlsanity.com/paper/2309.15082/) (2023)
- [Re-Evaluating LiDAR Scene Flow for Autonomous Driving](https://mlsanity.com/paper/2304.02150/) (2023)
- [Bring Event into RGB and LiDAR: Hierarchical Visual-Motion Fusion for Scene Flow](https://mlsanity.com/paper/2403.07432/) (2024)
- [DELFlow: Dense Efficient Learning of Scene Flow for Large-Scale Point Clouds](https://mlsanity.com/paper/2308.04383/) (2023)
- [SVQNet: Sparse Voxel-Adjacent Query Network for 4D Spatio-Temporal LiDAR Semantic Segmentation](https://mlsanity.com/paper/2308.13323/) (2023)
- [Occlusion Guided Self-supervised Scene Flow Estimation on 3D Point Clouds](https://mlsanity.com/paper/2104.04724/) (2021)
- [Unsupervised confidence for LiDAR depth maps and applications](https://mlsanity.com/paper/2210.03118/) (2022)
- [Learning Optical Flow and Scene Flow with Bidirectional Camera-LiDAR Fusion](https://mlsanity.com/paper/2303.12017/) (2023)
- [Dynamic Semantic Occupancy Mapping using 3D Scene Flow and Closed-Form Bayesian Inference](https://mlsanity.com/paper/2108.03180/) (2021)
- [DVMN: Dense Validity Mask Network for Depth Completion](https://mlsanity.com/paper/2107.06709/) (2021)
- [Attentive Multimodal Fusion for Optical and Scene Flow](https://mlsanity.com/paper/2307.15301/) (2023)
- [FedRSU: Federated Learning for Scene Flow Estimation on Roadside Units](https://mlsanity.com/paper/2401.12862/) (2024)
- [DEFLOW: Self-supervised 3D Motion Estimation of Debris Flow](https://mlsanity.com/paper/2304.02569/) (2023)
- [RMS-FlowNet++: Efficient and Robust Multi-Scale Scene Flow Estimation for Large-Scale Point Clouds](https://mlsanity.com/paper/2407.01129/) (2024)
- [Learning from Spatio-temporal Correlation for Semi-Supervised LiDAR Semantic Segmentation](https://mlsanity.com/paper/2410.06893/) (2024)
- [Iterative Camera-LiDAR Extrinsic Optimization via Surrogate Diffusion](https://mlsanity.com/paper/2411.10936/) (2024)
- [INV-Flow2PoseNet: Light-Resistant Rigid Object Pose from Optical Flow of RGB-D Images using Images, Normals and Vertices](https://mlsanity.com/paper/2209.06562/) (2022)
- [D-PLS: Decoupled Semantic Segmentation for 4D-Panoptic-LiDAR-Segmentation](https://mlsanity.com/paper/2501.15870/) (2025)
- [Iterative Camera-LiDAR Extrinsic Optimization via Surrogate Diffusion](https://mlsanity.com/paper/2506.14706/) (2025)
- [SF3D-RGB: Scene Flow Estimation from Monocular Camera and Sparse LiDAR](https://mlsanity.com/paper/2602.21699/) (2026)

## Abstract (arXiv)

Scene flow is the dense 3D reconstruction of motion and geometry of a scene. Most state-of-the-art methods use a pair of stereo images as input for full scene reconstruction. These methods depend a lot on the quality of the RGB images and perform poorly in regions with reflective objects, shadows, ill-conditioned light environment and so on. LiDAR measurements are much less sensitive to the aforementioned conditions but LiDAR features are in general unsuitable for matching tasks due to their sparse nature. Hence, using both LiDAR and RGB can potentially overcome the individual disadvantages of each sensor by mutual improvement and yield robust features which can improve the matching process. In this paper, we present DeepLiDARFlow, a novel deep learning architecture which fuses high level RGB and LiDAR features at multiple scales in a monocular setup to predict dense scene flow. Its performance is much better in the critical regions where image-only and LiDAR-only methods are inaccurate. We verify our DeepLiDARFlow using the established data sets KITTI and FlyingThings3D and we show strong robustness compared to several state-of-the-art methods which used other input modalities. The code of our paper is available at https://github.com/dfki-av/DeepLiDARFlow.