# SceneTracker: Long-term Scene Flow Estimation Network

arXiv: https://arxiv.org/abs/2403.19924 · HTML: https://mlsanity.com/paper/2403.19924/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 36 references that are in the map.
- First posted on arXiv 29 Mar 2024.

## Builds on (most cited first, 27 of 27)

- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [Scalability in Perception for Autonomous Driving: Waymo Open Dataset](https://mlsanity.com/paper/1912.04838/) (2019)
- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [4D Gaussian Splatting for Real-Time Dynamic Scene Rendering](https://mlsanity.com/paper/2310.08528/) (2023)
- [Center-based 3D Object Detection and Tracking](https://mlsanity.com/paper/2006.11275/) (2020)
- [FlowNet: Learning Optical Flow with Convolutional Networks](https://mlsanity.com/paper/1504.06852/) (2015)
- [PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume](https://mlsanity.com/paper/1709.02371/) (2017)
- [A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimation](https://mlsanity.com/paper/1512.02134/) (2015)
- [Dynamic 3D Gaussians: Tracking by Persistent Dynamic View Synthesis](https://mlsanity.com/paper/2308.09713/) (2023)
- [CoTracker: It is Better to Track Together](https://mlsanity.com/paper/2307.07635/) (2023)
- [TAPIR: Tracking Any Point with per-frame Initialization and temporal Refinement](https://mlsanity.com/paper/2306.08637/) (2023)
- [PointOdyssey: A Large-Scale Synthetic Dataset for Long-Term Point Tracking](https://mlsanity.com/paper/2307.15055/) (2023)
- [Unifying Flow, Stereo and Depth Estimation](https://mlsanity.com/paper/2211.05783/) (2022)
- [TAP-Vid: A Benchmark for Tracking Any Point in a Video](https://mlsanity.com/paper/2211.03726/) (2022)
- [FlowNet3D: Learning Scene Flow in 3D Point Clouds](https://mlsanity.com/paper/1806.01411/) (2018)
- [Learning to Estimate Hidden Motions with Global Motion Aggregation](https://mlsanity.com/paper/2104.02409/) (2021)
- [Particle Video Revisited: Tracking Through Occlusions Using Point Trajectories](https://mlsanity.com/paper/2204.04153/) (2022)
- [Hierarchical Neural Architecture Search for Deep Stereo Matching](https://mlsanity.com/paper/2010.13501/) (2020)
- [RAFT-3D: Scene Flow using Rigid-Motion Embeddings](https://mlsanity.com/paper/2012.00726/) (2020)
- [SimpleTrack: Understanding and Rethinking 3D Multi-object Tracking](https://mlsanity.com/paper/2111.09621/) (2021)
- [MFT: Long-Term Tracking of Every Pixel](https://mlsanity.com/paper/2305.12998/) (2023)
- [CamLiFlow: Bidirectional Camera-LiDAR Fusion for Joint Optical Flow and Scene Flow Estimation](https://mlsanity.com/paper/2111.10502/) (2021)
- [Learning to Segment Rigid Motions from Two Frames](https://mlsanity.com/paper/2101.03694/) (2021)
- [DriveTrack: A Benchmark for Long-Range Point Tracking in Real-World Videos](https://mlsanity.com/paper/2312.09523/) (2023)
- [TrajectoryFormer: 3D Object Tracking Transformer with Predictive Trajectory Hypotheses](https://mlsanity.com/paper/2306.05888/) (2023)
- [SplatFlow: Learning Multi-frame Optical Flow via Splatting](https://mlsanity.com/paper/2306.08887/) (2023)

## Built on by (most cited first, 26 of 26)

- [SpatialTrackerV2: 3D Point Tracking Made Easy](https://mlsanity.com/paper/2507.12462/) (2025)
- [TAPVid-3D: A Benchmark for Tracking Any Point in 3D](https://mlsanity.com/paper/2407.05921/) (2024)
- [TAPIP3D: Tracking Any Point in Persistent 3D Geometry](https://mlsanity.com/paper/2504.14717/) (2025)
- [DELTA: Dense Efficient Long-range 3D Tracking for any video](https://mlsanity.com/paper/2410.24211/) (2024)
- [Dynamic Point Maps: A Versatile Representation for Dynamic 3D Reconstruction](https://mlsanity.com/paper/2503.16318/) (2025)
- [Any4D: Unified Feed-Forward Metric 4D Reconstruction](https://mlsanity.com/paper/2512.10935/) (2025)
- [Multi-View 3D Point Tracking](https://mlsanity.com/paper/2508.21060/) (2025)
- [Reconstructing 4D Spatial Intelligence: A Survey](https://mlsanity.com/paper/2507.21045/) (2025)
- [CoWTracker: Tracking by Warping instead of Correlation](https://mlsanity.com/paper/2602.04877/) (2026)
- [DELTAv2: Accelerating Dense 3D Tracking](https://mlsanity.com/paper/2508.01170/) (2025)
- [TrackingWorld: World-centric Monocular 3D Tracking of Almost All Pixels](https://mlsanity.com/paper/2512.08358/) (2025)
- [SeeU: Seeing the Unseen World via 4D Dynamics-aware Generation](https://mlsanity.com/paper/2512.03350/) (2025)
- [Video Understanding: From Geometry and Semantics to Unified Models](https://mlsanity.com/paper/2603.17840/) (2026)
- [TrackCraft3R: Repurposing Video Diffusion Transformers for Dense 3D Tracking](https://mlsanity.com/paper/2605.12587/) (2026)
- [SM4RT: Learning Structured Motion Geometry for 4D Reconstruction](https://mlsanity.com/paper/2607.22534/) (2026)
- [SynthVerse: A Large-Scale Diverse Synthetic Dataset for Point Tracking](https://mlsanity.com/paper/2602.04441/) (2026)
- [MotionCrafter: Dense Geometry and Motion Reconstruction with a 4D VAE](https://mlsanity.com/paper/2602.08961/) (2026)
- [Is This Tracker On? A Benchmark Protocol for Dynamic Tracking](https://mlsanity.com/paper/2510.19819/) (2025)
- [Moaw: Unleashing Motion Awareness for Video Diffusion Models](https://mlsanity.com/paper/2601.12761/) (2026)
- [TAPVid-MV: A Benchmark for Tracking Any Point in 3D Across Multiple Views](https://mlsanity.com/paper/2609.01899/) (2026)
- [Point4D: Long-range 4D Motion Reconstruction](https://mlsanity.com/paper/2609.09145/) (2026)
- [TAPVid-360: Tracking Any Point in 360 from Narrow Field of View Video](https://mlsanity.com/paper/2511.21946/) (2025)
- [Geometry OR Tracker: Universal Geometric Operating Room Tracking](https://mlsanity.com/paper/2603.00560/) (2026)
- [PCSTracker: Long-Term Scene Flow Estimation for Point Cloud Sequences](https://mlsanity.com/paper/2603.19762/) (2026)
- [Velox: Learning Representations of 4D Geometry and Appearance](https://mlsanity.com/paper/2605.04527/) (2026)
- [MoSE3: Learning World-Space SE(3) at Every Pixel](https://mlsanity.com/paper/2610.03716/) (2026)

## Abstract (arXiv)

Considering that scene flow estimation has the capability of the spatial domain to focus but lacks the coherence of the temporal domain, this study proposes long-term scene flow estimation (LSFE), a comprehensive task that can simultaneously capture the fine-grained and long-term 3D motion in an online manner. We introduce SceneTracker, the first LSFE network that adopts an iterative approach to approximate the optimal 3D trajectory. The network dynamically and simultaneously indexes and constructs appearance correlation and depth residual features. Transformers are then employed to explore and utilize long-range connections within and between trajectories. With detailed experiments, SceneTracker shows superior capabilities in addressing 3D spatial occlusion and depth noise interference, highly tailored to the needs of the LSFE task. We build a real-world evaluation dataset, LSFDriving, for the LSFE field and use it in experiments to further demonstrate the advantage of SceneTracker in generalization abilities. The code and data are available at https://github.com/wwsource/SceneTracker.