# ClusterVO: Clustering Moving Instances and Estimating Visual Odometry for Self and Surroundings

arXiv: https://arxiv.org/abs/2003.12980 · HTML: https://mlsanity.com/paper/2003.12980/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Mar 2020.

## Built on by (most cited first, 17 of 17)

- [ParticleSfM: Exploiting Dense Point Trajectories for Localizing Moving Cameras in the Wild](https://mlsanity.com/paper/2207.09137/) (2022)
- [MultiBodySync: Multi-Body Segmentation and Motion Estimation via 3D Scan Synchronization](https://mlsanity.com/paper/2101.06605/) (2021)
- [Multiway Non-rigid Point Cloud Registration via Learned Functional Map Synchronization](https://mlsanity.com/paper/2111.12878/) (2021)
- [D3FlowSLAM: Self-Supervised Dynamic SLAM with Flow Motion Decomposition and DINO Guidance](https://mlsanity.com/paper/2207.08794/) (2022)
- [MOTSLAM: MOT-assisted monocular dynamic SLAM using single-view depth estimation](https://mlsanity.com/paper/2210.02038/) (2022)
- [Semantic Visual Simultaneous Localization and Mapping: A Survey](https://mlsanity.com/paper/2209.06428/) (2022)
- [S3LAM: Structured Scene SLAM](https://mlsanity.com/paper/2109.07339/) (2021)
- [View Birdification in the Crowd: Ground-Plane Localization from Perceived Movements](https://mlsanity.com/paper/2111.05060/) (2021)
- [TwistSLAM: Constrained SLAM in Dynamic Environment](https://mlsanity.com/paper/2202.12384/) (2022)
- [VGF-Net: Visual-Geometric Fusion Learning for Simultaneous Drone Navigation and Height Mapping](https://mlsanity.com/paper/2104.03109/) (2021)
- [ViewBirdiformer: Learning to recover ground-plane crowd trajectories and ego-motion from a single ego-centric view](https://mlsanity.com/paper/2210.06332/) (2022)
- [GSLAMOT: A Tracklet and Query Graph-based Simultaneous Locating, Mapping, and Multiple Object Tracking System](https://mlsanity.com/paper/2408.09191/) (2024)
- [Visual-Inertial Multi-Instance Dynamic SLAM with Object-level Relocalisation](https://mlsanity.com/paper/2208.04274/) (2022)
- [TwistSLAM++: Fusing multiple modalities for accurate dynamic semantic SLAM](https://mlsanity.com/paper/2209.07888/) (2022)
- [UniQuadric: A SLAM Backend for Unknown Rigid Object 3D Tracking and Light-Weight Modeling](https://mlsanity.com/paper/2309.17036/) (2023)
- [Jacobian Computation for Cumulative B-Splines on SE(3) and Application to Continuous-Time Object Tracking](https://mlsanity.com/paper/2201.10602/) (2022)
- [MoSE3: Learning World-Space SE(3) at Every Pixel](https://mlsanity.com/paper/2610.03716/) (2026)

## Abstract (arXiv)

We present ClusterVO, a stereo Visual Odometry which simultaneously clusters and estimates the motion of both ego and surrounding rigid clusters/objects. Unlike previous solutions relying on batch input or imposing priors on scene structure or dynamic object models, ClusterVO is online, general and thus can be used in various scenarios including indoor scene understanding and autonomous driving. At the core of our system lies a multi-level probabilistic association mechanism and a heterogeneous Conditional Random Field (CRF) clustering approach combining semantic, spatial and motion information to jointly infer cluster segmentations online for every frame. The poses of camera and dynamic objects are instantly solved through a sliding-window optimization. Our system is evaluated on Oxford Multimotion and KITTI dataset both quantitatively and qualitatively, reaching comparable results to state-of-the-art solutions on both odometry and dynamic trajectory recovery.