# CMU-Flownet: Exploring Point Cloud Scene Flow Estimation in Occluded Scenario

arXiv: https://arxiv.org/abs/2404.10571 · HTML: https://mlsanity.com/paper/2404.10571/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 27 references that are in the map.
- First posted on arXiv 16 Apr 2024.

## Builds on (most cited first, 21 of 21)

- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimation](https://mlsanity.com/paper/1512.02134/) (2015)
- [Point-GNN: Graph Neural Network for 3D Object Detection in a Point Cloud](https://mlsanity.com/paper/2003.01251/) (2020)
- [FlowNet3D: Learning Scene Flow in 3D Point Clouds](https://mlsanity.com/paper/1806.01411/) (2018)
- [FLOT: Scene Flow on Point Clouds Guided by Optimal Transport](https://mlsanity.com/paper/2007.11142/) (2020)
- [MaskFlownet: Asymmetric Feature Matching with Learnable Occlusion Mask](https://mlsanity.com/paper/2003.10955/) (2020)
- [HPLFlowNet: Hierarchical Permutohedral Lattice FlowNet for Scene Flow Estimation on Large-scale Point Clouds](https://mlsanity.com/paper/1906.05332/) (2019)
- [Semantic Video Segmentation by Gated Recurrent Flow Propagation](https://mlsanity.com/paper/1612.08871/) (2016)
- [PV-RAFT: Point-Voxel Correlation Fields for Scene Flow Estimation of Point Clouds](https://mlsanity.com/paper/2012.00987/) (2020)
- [FlowStep3D: Model Unrolling for Self-Supervised Scene Flow Estimation](https://mlsanity.com/paper/2011.10147/) (2020)
- [Learning Video Object Segmentation with Visual Memory](https://mlsanity.com/paper/1704.05737/) (2017)
- [PointPWC-Net: A Coarse-to-Fine Network for Supervised and Self-Supervised Scene Flow Estimation on 3D Point Clouds](https://mlsanity.com/paper/1911.12408/) (2019)
- [Bi-PointFlowNet: Bidirectional Learning for Point Cloud Based Scene Flow Estimation](https://mlsanity.com/paper/2207.07522/) (2022)
- [Hierarchical Attention Learning of Scene Flow in 3D Point Clouds](https://mlsanity.com/paper/2010.05762/) (2020)
- [What Matters for 3D Scene Flow Network](https://mlsanity.com/paper/2207.09143/) (2022)
- [FESTA: Flow Estimation via Spatial-Temporal Attention for Scene Point Clouds](https://mlsanity.com/paper/2104.00798/) (2021)
- [Self-Supervised 3D Scene Flow Estimation Guided by Superpoints](https://mlsanity.com/paper/2305.02528/) (2023)
- [PWOC-3D: Deep Occlusion-Aware End-to-End Scene Flow Estimation](https://mlsanity.com/paper/1904.06116/) (2019)
- [Occlusion Guided Scene Flow Estimation on 3D Point Clouds](https://mlsanity.com/paper/2011.14880/) (2020)
- [GMA3D: Local-Global Attention Learning to Estimate Occluded Motions of Scene Flow](https://mlsanity.com/paper/2210.03296/) (2022)

## Built on by (most cited first, 1 of 1)

- [Open 3D World in Autonomous Driving](https://mlsanity.com/paper/2408.10880/) (2024)

## Abstract (arXiv)

Occlusions hinder point cloud frame alignment in LiDAR data, a challenge inadequately addressed by scene flow models tested mainly on occlusion-free datasets. Attempts to integrate occlusion handling within networks often suffer accuracy issues due to two main limitations: a) the inadequate use of occlusion information, often merging it with flow estimation without an effective integration strategy, and b) reliance on distance-weighted upsampling that falls short in correcting occlusion-related errors. To address these challenges, we introduce the Correlation Matrix Upsampling Flownet (CMU-Flownet), incorporating an occlusion estimation module within its cost volume layer, alongside an Occlusion-aware Cost Volume (OCV) mechanism. Specifically, we propose an enhanced upsampling approach that expands the sensory field of the sampling process which integrates a Correlation Matrix designed to evaluate point-level similarity. Meanwhile, our model robustly integrates occlusion data within the context of scene flow, deploying this information strategically during the refinement phase of the flow estimation. The efficacy of this approach is demonstrated through subsequent experimental validation. Empirical assessments reveal that CMU-Flownet establishes state-of-the-art performance within the realms of occluded Flyingthings3D and KITTY datasets, surpassing previous methodologies across a majority of evaluated metrics.