# Robust Multimodal Dynamic Object Segmentation

arXiv: https://arxiv.org/abs/2607.18153 · HTML: https://mlsanity.com/paper/2607.18153/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 22 of its 31 references that are in the map.
- First posted on arXiv 20 Jul 2026.

## Builds on (most cited first, 22 of 22)

- [SAM 2: Segment Anything in Images and Videos](https://mlsanity.com/paper/2408.00714/) (2024)
- [Vision Transformers for Dense Prediction](https://mlsanity.com/paper/2103.13413/) (2021)
- [DUSt3R: Geometric 3D Vision Made Easy](https://mlsanity.com/paper/2312.14132/) (2023)
- [MonST3R: A Simple Approach for Estimating Geometry in the Presence of Motion](https://mlsanity.com/paper/2410.03825/) (2024)
- [PointOdyssey: A Large-Scale Synthetic Dataset for Long-Term Point Tracking](https://mlsanity.com/paper/2307.15055/) (2023)
- [SEA-RAFT: Simple, Efficient, Accurate RAFT for Optical Flow](https://mlsanity.com/paper/2405.14793/) (2024)
- [Learning Two-View Correspondences and Geometry Using Order-Aware Network](https://mlsanity.com/paper/1908.04964/) (2019)
- [Self-supervised Video Object Segmentation by Motion Grouping](https://mlsanity.com/paper/2104.07658/) (2021)
- [DynaSLAM: Tracking, Mapping and Inpainting in Dynamic Scenes](https://mlsanity.com/paper/1806.05620/) (2018)
- [CoTracker3: Simpler and Better Point Tracking by Pseudo-Labelling Real Videos](https://mlsanity.com/paper/2410.11831/) (2024)
- [ParticleSfM: Exploiting Dense Point Trajectories for Localizing Moving Cameras in the Wild](https://mlsanity.com/paper/2207.09137/) (2022)
- [Easi3R: Estimating Disentangled Motion from DUSt3R Without Training](https://mlsanity.com/paper/2503.24391/) (2025)
- [Segmenting Moving Objects via an Object-Centric Layered Representation](https://mlsanity.com/paper/2207.02206/) (2022)
- [Motion-Attentive Transition for Zero-Shot Video Object Segmentation](https://mlsanity.com/paper/2003.04253/) (2020)
- [LEAP-VO: Long-term Effective Any Point Tracking for Visual Odometry](https://mlsanity.com/paper/2401.01887/) (2024)
- [DAS3R: Dynamics-Aware Gaussian Splatting for Static Scene Reconstruction](https://mlsanity.com/paper/2412.19584/) (2024)
- [RoMo: Robust Motion Segmentation Improves Structure from Motion](https://mlsanity.com/paper/2411.18650/) (2024)
- [Bootstrapping Objectness from Videos by Relaxed Common Fate and Visual Grouping](https://mlsanity.com/paper/2304.08025/) (2023)
- [Robust 3D Gaussian Splatting for Novel View Synthesis in Presence of Distractors](https://mlsanity.com/paper/2408.11697/) (2024)
- [Learning segmentation from point trajectories](https://mlsanity.com/paper/2501.12392/) (2025)
- [SPARS3R: Semantic Prior Alignment and Regularization for Sparse 3D Reconstruction](https://mlsanity.com/paper/2411.12592/) (2024)
- [DymSLAM:4D Dynamic Scene Reconstruction Based on Geometrical Motion Segmentation](https://mlsanity.com/paper/2003.04569/) (2020)

## Abstract (arXiv)

Dynamic object segmentation plays a critical role in many visual applications such as static scene reconstruction from dynamic videos. However, existing optical flow-based methods fail to ensure consistent static/dynamic segmentation along object boundaries, while 3D reconstruction-based approaches are highly sensitive to reconstruction errors. To address these limitations, we present a dynamic object segmentation framework that can generate both precise and complete dynamic masks by integrating multimodal cues including 2D point tracks, 3D reconstruction, and semantic information. We design a network combining Transformer architectures with feature clustering aggregation modules to perform static/dynamic classification of multimodal feature trajectories. It enables the model to adaptively determine which type of feature should dominate based on the characteristics of each scene, while also mitigating the impact of feature degradation. Additionally, we introduce a novel point-query-based SAM post-processing method capable of handling multiple objects within a single mask. Extensive experiments demonstrate that our approach achieves state-of-the-art performance in both dynamic object segmentation and static scene reconstruction tasks.