# SF2SE3: Clustering Scene Flow into SE(3)-Motions via Proposal and Selection

arXiv: https://arxiv.org/abs/2209.08532 · HTML: https://mlsanity.com/paper/2209.08532/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 31 references that are in the map.
- First posted on arXiv 18 Sep 2022.

## Builds on (most cited first, 11 of 11)

- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimation](https://mlsanity.com/paper/1512.02134/) (2015)
- [FlowNet3D: Learning Scene Flow in 3D Point Clouds](https://mlsanity.com/paper/1806.01411/) (2018)
- [Hierarchical Neural Architecture Search for Deep Stereo Matching](https://mlsanity.com/paper/2010.13501/) (2020)
- [RAFT-3D: Scene Flow using Rigid-Motion Embeddings](https://mlsanity.com/paper/2012.00726/) (2020)
- [Self-Supervised Monocular Scene Flow Estimation](https://mlsanity.com/paper/2004.04143/) (2020)
- [CamLiFlow: Bidirectional Camera-LiDAR Fusion for Joint Optical Flow and Scene Flow Estimation](https://mlsanity.com/paper/2111.10502/) (2021)
- [Deep Rigid Instance Scene Flow](https://mlsanity.com/paper/1904.08913/) (2019)
- [Learning to Segment Rigid Motions from Two Frames](https://mlsanity.com/paper/2101.03694/) (2021)
- [SENSE: a Shared Encoder Network for Scene-flow Estimation](https://mlsanity.com/paper/1910.12361/) (2019)
- [A Deep Temporal Fusion Framework for Scene Flow Using a Learnable Motion Model and Occlusions](https://mlsanity.com/paper/2011.01603/) (2020)

## Built on by (most cited first, 6 of 6)

- [GMSF: Global Matching Scene Flow](https://mlsanity.com/paper/2305.17432/) (2023)
- [Scene-Centric Unsupervised Panoptic Segmentation](https://mlsanity.com/paper/2504.01955/) (2025)
- [DiffSF: Diffusion Models for Scene Flow Estimation](https://mlsanity.com/paper/2403.05327/) (2024)
- [Motion-Refined DINOSAUR for Unsupervised Multi-Object Discovery](https://mlsanity.com/paper/2509.02545/) (2025)
- [MS-RAFT-3D: A Multi-Scale Architecture for Recurrent Image-Based Scene Flow](https://mlsanity.com/paper/2506.01443/) (2025)
- [Scene-Centric Unsupervised Video Panoptic Segmentation](https://mlsanity.com/paper/2606.04925/) (2026)

## Abstract (arXiv)

We propose SF2SE3, a novel approach to estimate scene dynamics in form of a segmentation into independently moving rigid objects and their SE(3)-motions. SF2SE3 operates on two consecutive stereo or RGB-D images. First, noisy scene flow is obtained by application of existing optical flow and depth estimation algorithms. SF2SE3 then iteratively (1) samples pixel sets to compute SE(3)-motion proposals, and (2) selects the best SE(3)-motion proposal with respect to a maximum coverage formulation. Finally, objects are formed by assigning pixels uniquely to the selected SE(3)-motions based on consistency with the input scene flow and spatial proximity. The main novelties are a more informed strategy for the sampling of motion proposals and a maximum coverage formulation for the proposal selection. We conduct evaluations on multiple datasets regarding application of SF2SE3 for scene flow estimation, object segmentation and visual odometry. SF2SE3 performs on par with the state of the art for scene flow estimation and is more accurate for segmentation and odometry.