mlsanityData to 14 Oct 2026

CamLiFlow: Bidirectional Camera-LiDAR Fusion for Joint Optical Flow and Scene Flow Estimation

arXiv 2111.10502 ↗First posted 20 Nov 2021cs.CVcited by 56

Not on the map. Why?

Abstract (arXiv)

In this paper, we study the problem of jointly estimating the optical flow and scene flow from synchronized 2D and 3D data. Previous methods either employ a complex pipeline that splits the joint task into independent stages, or fuse 2D and 3D information in an "early-fusion" or "late-fusion" manner. Such one-size-fits-all approaches suffer from a dilemma of failing to fully utilize the characteristic of each modality or to maximize the inter-modality complementarity. To address the problem, we propose a novel end-to-end framework, called CamLiFlow. It consists of 2D and 3D branches with multiple bidirectional connections between them in specific layers. Different from previous work, we apply a point-based 3D branch to better extract the geometric features and design a symmetric learnable operator to fuse dense image features and sparse point features. Experiments show that CamLiFlow achieves better performance with fewer parameters. Our method ranks 1st on the KITTI Scene Flow benchmark, outperforming the previous art with 1/7 parameters. Code is available at https://github.com/MCG-NJU/CamLiFlow.

Key facts. Cited by 56 papers in the map (data through 14 Oct 2026). Builds on 43 of its 62 references that are in the map. First posted on arXiv 20 Nov 2021.

Builds on

43 of 62 references.

References · 43

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  4. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 7,005
  5. 2016Densely Connected Convolutional Networkscited by 6,720
  6. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,248
  7. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,444
  8. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,401
  9. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,897
  10. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,415
  11. 2015FlowNet: Learning Optical Flow with Convolutional Networkscited by 1,059
  12. 2017PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volumecited by 968
31 more
  1. 2015A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimationcited by 881
  2. 2016FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networkscited by 784
  3. 2016Multi-View 3D Object Detection Network for Autonomous Drivingcited by 724
  4. 2018PointConv: Deep Convolutional Networks on 3D Point Cloudscited by 657
  5. 2017Frustum PointNets for 3D Object Detection from RGB-D Datacited by 640
  6. 2019PointPainting: Sequential Fusion for 3D Object Detectioncited by 449
  7. 2016Optical Flow Estimation using a Spatial Pyramid Networkcited by 424
  8. 2020Deep Continuous Fusion for Multi-Sensor 3D Object Detectioncited by 263
  9. 2019GA-Net: Guided Aggregation Net for End-to-end Stereo Matchingcited by 244
  10. 2018FlowNet3D: Learning Scene Flow in 3D Point Cloudscited by 242
  11. 2019Pseudo-LiDAR++: Accurate Depth for 3D Object Detection in Autonomous Drivingcited by 188
  12. 2017PointFusion: Deep Sensor Fusion for 3D Bounding Box Estimationcited by 170
  13. 2017Sparse-to-Dense: Depth Prediction from Sparse Depth Samples and a Single Imagecited by 161
  14. 2019Iterative Residual Refinement for Joint Optical Flow and Occlusion Estimationcited by 131
  15. 2019Neural-Guided RANSAC: Learning Where to Sample Model Hypothesescited by 129
  16. 2020FLOT: Scene Flow on Point Clouds Guided by Optimal Transportcited by 129
  17. 2020RAFT-3D: Scene Flow using Rigid-Motion Embeddingscited by 122
  18. 2019MeteorNet: Deep Learning on Dynamic 3D Point Cloud Sequencescited by 117
  19. 2019HPLFlowNet: Hierarchical Permutohedral Lattice FlowNet for Scene Flow Estimation on Large-scale Point Cloudscited by 109
  20. 2021Deep Dual-resolution Networks for Real-time and Accurate Semantic Segmentation of Road Scenescited by 89
  21. 2020PV-RAFT: Point-Voxel Correlation Fields for Scene Flow Estimation of Point Cloudscited by 74
  22. 2020FlowStep3D: Model Unrolling for Self-Supervised Scene Flow Estimationcited by 71
  23. 2019FlowNet3D++: Geometric Losses For Deep Scene Flow Estimationcited by 55
  24. 2019Deep Rigid Instance Scene Flowcited by 54
  25. 2021Learning to Segment Rigid Motions from Two Framescited by 46
  26. 2019SENSE: a Shared Encoder Network for Scene-flow Estimationcited by 38
  27. 2019PointPWC-Net: A Coarse-to-Fine Network for Supervised and Self-Supervised Scene Flow Estimation on 3D Point Cloudscited by 38
  28. 2020DeepLiDARFlow: A Deep Learning Architecture For Scene Flow Estimation Using Monocular Camera and Sparse LiDARcited by 24
  29. 2021Binary TTC: A Temporal Geofence for Autonomous Navigationcited by 20
  30. 2021Advancing Self-supervised Monocular Depth Learning with Sparse LiDARcited by 13
  31. 2019LiDAR-Flow: Dense Scene Flow Estimation from Sparse LiDAR and Stereo Imagescited by 12

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 3; 2023: 18; 2024: 19; 2025: 9; 2026: 7.

56 papers.

Sort:

By area:

  1. 2023RoMa: Robust Dense Feature Matchingcited by 302
  2. 2023Spring: A High-Resolution High-Detail Dataset and Benchmark for Scene Flow, Optical Flow and Stereocited by 172
  3. 2023Delivering Arbitrary-Modal Semantic Segmentationcited by 117
  4. 2023Fully Sparse 3D Occupancy Predictioncited by 64
  5. 2023MUVO: A Multimodal Generative World Model for Autonomous Driving with Geometric Representationscited by 33
  6. 2024Learning Modality-agnostic Representation for Semantic Segmentation from Any Modalitiescited by 31
  7. 2022PanoFlow: Learning 360deg Optical Flow for Surrounding Temporal Understandingcited by 26
  8. 2023RPEFlow: Multimodal Fusion of RGB-PointCloud-Event for Joint Optical Flow and Scene Flow Estimationcited by 26
  9. 2024DVLO: Deep Visual-LiDAR Odometry with Local-to-Global Feature Fusion and Bi-Directional Structure Alignmentcited by 26
  10. 2024SceneTracker: Long-term Scene Flow Estimation Networkcited by 26
  11. 2023Self-Supervised 3D Scene Flow Estimation Guided by Superpointscited by 24
  12. 2024A Survey on Deep Stereo Matching in the Twentiescited by 23
38 more
  1. 2024Centering the Value of Every Modality: Towards Efficient and Resilient Modality-agnostic Semantic Segmentationcited by 22
  2. 2025Zero-Shot Monocular Scene Flow Estimation in the Wildcited by 22
  3. 2025Multi-View 3D Point Trackingcited by 17
  4. 2023CVRecon: Rethinking 3D Geometric Feature Learning For Neural Reconstructioncited by 16
  5. 2023CCMR: High Resolution Optical Flow Estimation via Coarse-to-Fine Context-Guided Motion Reasoningcited by 15
  6. 2024Bring Event into RGB and LiDAR: Hierarchical Visual-Motion Fusion for Scene Flowcited by 15
  7. 2024DeepInteraction++: Multi-Modality Interaction for Autonomous Drivingcited by 15
  8. 2022M-FUSE: Multi-frame Fusion for Scene Flow Estimationcited by 14
  9. 2023GMSF: Global Matching Scene Flowcited by 14
  10. 2023DELFlow: Dense Efficient Learning of Scene Flow for Large-Scale Point Cloudscited by 14
  11. 2024BlinkVision: A Benchmark for Optical Flow, Scene Flow and Point Tracking Estimation using RGB Frames and Eventscited by 13
  12. 2023Density-invariant Features for Distant Point Cloud Registrationcited by 12
  13. 2024MAGIC++: Efficient and Resilient Modality-Agnostic Semantic Segmentation via Hierarchical Modality Selectioncited by 12
  14. 2023Unleash the Potential of Image Branch for Cross-modal 3D Object Detectioncited by 11
  15. 2023Learning Optical Flow and Scene Flow with Bidirectional Camera-LiDAR Fusioncited by 10
  16. 2023ADFactory: An Effective Framework for Generalizing Optical Flow with Nerfcited by 9
  17. 2025Reducing Unimodal Bias in Multi-Modal Semantic Segmentation with Multi-Scale Functional Entropy Regularizationcited by 9
  18. 2024Multi-Model 3D Registration: Finding Multiple Moving Objects in Cluttered Point Cloudscited by 7
  19. 2022SF2SE3: Clustering Scene Flow into SE(3)-Motions via Proposal and Selectioncited by 6
  20. 2024Part-Whole Relational Fusion Towards Multi-Modal Scene Understandingcited by 6
  21. 2024OptFlow: Fast Optimization-based Scene Flow Estimation without Supervisioncited by 5
  22. 2024Self-Supervised Bird's Eye View Motion Prediction with Cross-Modality Signalscited by 5
  23. 2023Image-based Geolocalization by Ground-to-2.5D Map Matchingcited by 4
  24. 2023Attentive Multimodal Fusion for Optical and Scene Flowcited by 3
  25. 2024ScaleFlow++: Robust and Accurate Estimation of 3D Motion from Videocited by 3
  26. 2024Drone Stereo Vision for Radiata Pine Branch Detection and Distance Measurement: Integrating SGBM and Segmentation Modelscited by 3
  27. 2024Self-Assessed Generation: Trustworthy Label Generation for Optical Flow and Stereo Matching in Real-worldcited by 3
  28. 2025JointSplat: Probabilistic Joint Flow-Depth Optimization for Sparse-View Gaussian Splattingcited by 3
  29. 2025RaLiFlow: Scene Flow Estimation with 4D Radar and LiDAR Point Cloudscited by 3
  30. 2024NeRF-DetS: Enhanced Adaptive Spatial-wise Sampling and View-wise Fusion Strategies for NeRF-based Indoor Multi-view 3D Object Detectioncited by 2
  31. 2025EMoTive: Event-guided Trajectory Modeling for 3D Motion Estimationcited by 2
  32. 2025MS-RAFT-3D: A Multi-Scale Architecture for Recurrent Image-Based Scene Flowcited by 2
  33. 2023DEFLOW: Self-supervised 3D Motion Estimation of Debris Flowcited by 1
  34. 2024Iterative Camera-LiDAR Extrinsic Optimization via Surrogate Diffusioncited by 1
  35. 2025High-quality Pseudo-labeling for Point Cloud Segmentation with Scene-level Annotationcited by 1
  36. 2026x2-Fusion: Cross-Modality and Cross-Dimension Flow Estimation in Event Edge Spacecited by 1
  37. 2024ScaleFlow++: Robust and Accurate Estimation of 3D Motion from Videocited by 0
  38. 2025Iterative Camera-LiDAR Extrinsic Optimization via Surrogate Diffusioncited by 0

50 most cited of 56 All 56 (JSON)