# CoWTracker: Tracking by Warping instead of Correlation

arXiv: https://arxiv.org/abs/2602.04877 · HTML: https://mlsanity.com/paper/2602.04877/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 47 references that are in the map.
- First posted on arXiv 4 Feb 2026.

## Builds on (most cited first, 30 of 36)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [Vision Transformers for Dense Prediction](https://mlsanity.com/paper/2103.13413/) (2021)
- [ViViT: A Video Vision Transformer](https://mlsanity.com/paper/2103.15691/) (2021)
- [VGGT: Visual Geometry Grounded Transformer](https://mlsanity.com/paper/2503.11651/) (2025)
- [Spatial Transformer Networks](https://mlsanity.com/paper/1506.02025/) (2015)
- [FlowNet: Learning Optical Flow with Convolutional Networks](https://mlsanity.com/paper/1504.06852/) (2015)
- [PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume](https://mlsanity.com/paper/1709.02371/) (2017)
- [CoTracker: It is Better to Track Together](https://mlsanity.com/paper/2307.07635/) (2023)
- [FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precision](https://mlsanity.com/paper/2407.08608/) (2024)
- [π3: Permutation-Equivariant Visual Geometry Learning](https://mlsanity.com/paper/2507.13347/) (2025)
- [A Short Note on the Kinetics-700 Human Action Dataset](https://mlsanity.com/paper/1907.06987/) (2019)
- [TAPIR: Tracking Any Point with per-frame Initialization and temporal Refinement](https://mlsanity.com/paper/2306.08637/) (2023)
- [PointOdyssey: A Large-Scale Synthetic Dataset for Long-Term Point Tracking](https://mlsanity.com/paper/2307.15055/) (2023)
- [TAP-Vid: A Benchmark for Tracking Any Point in a Video](https://mlsanity.com/paper/2211.03726/) (2022)
- [Visual Geometry Grounded Deep Structure From Motion](https://mlsanity.com/paper/2312.04563/) (2023)
- [SEA-RAFT: Simple, Efficient, Accurate RAFT for Optical Flow](https://mlsanity.com/paper/2405.14793/) (2024)
- [Spring: A High-Resolution High-Detail Dataset and Benchmark for Scene Flow, Optical Flow and Stereo](https://mlsanity.com/paper/2303.01943/) (2023)
- [BootsTAP: Bootstrapped Training for Tracking-Any-Point](https://mlsanity.com/paper/2402.00847/) (2024)
- [End-to-end representation learning for Correlation Filter based tracking](https://mlsanity.com/paper/1704.06036/) (2017)
- [RoboTAP: Tracking Arbitrary Points for Few-Shot Visual Imitation](https://mlsanity.com/paper/2308.15975/) (2023)
- [Local All-Pair Correspondence for Point Tracking](https://mlsanity.com/paper/2407.15420/) (2024)
- [TAPVid-3D: A Benchmark for Tracking Any Point in 3D](https://mlsanity.com/paper/2407.05921/) (2024)
- [TAPIP3D: Tracking Any Point in Persistent 3D Geometry](https://mlsanity.com/paper/2504.14717/) (2025)
- [DELTA: Dense Efficient Long-range 3D Tracking for any video](https://mlsanity.com/paper/2410.24211/) (2024)
- [Dense Optical Tracking: Connecting the Dots](https://mlsanity.com/paper/2312.00786/) (2023)
- [AllTracker: Efficient Dense Point Tracking at High Resolution](https://mlsanity.com/paper/2506.07310/) (2025)
- [TAPTR: Tracking Any Point with Transformers as Detection](https://mlsanity.com/paper/2403.13042/) (2024)
- [TAPNext: Tracking Any Point (TAP) as Next Token Prediction](https://mlsanity.com/paper/2504.05579/) (2025)

## Built on by (most cited first, 14 of 14)

- [MegaFlow: Zero-Shot Large Displacement Optical Flow](https://mlsanity.com/paper/2603.25739/) (2026)
- [SM4RT: Learning Structured Motion Geometry for 4D Reconstruction](https://mlsanity.com/paper/2607.22534/) (2026)
- [Causally Debiased Latent Action Model for Embodied Action Conditioned World Models](https://mlsanity.com/paper/2607.09185/) (2026)
- [Learning Global Motion with Compact Gaussians for Feed-Forward 4D Reconstruction](https://mlsanity.com/paper/2605.31595/) (2026)
- [Fast 4D Mesh Generation by Spatio-Temporal Attention Chains](https://mlsanity.com/paper/2605.19786/) (2026)
- [TAPVid-MV: A Benchmark for Tracking Any Point in 3D Across Multiple Views](https://mlsanity.com/paper/2609.01899/) (2026)
- [MachEmbodied-U0: Unified Understanding and Generation Model for Embodied Intelligence](https://mlsanity.com/paper/2609.25627/) (2026)
- [InternW0-Δ: A World Action Model Bridging Predictive Dynamics and Actions with 20K+ Hours of Open Data](https://mlsanity.com/paper/2609.31394/) (2026)
- [TrackEverything: Long Horizon Dense Tracking via De-Duplicating 3D Scene Representations](https://mlsanity.com/paper/2609.30222/) (2026)
- [E-WAVE: Event-based Continuous Optical Flow via Warping-Aligned Visual Encoding](https://mlsanity.com/paper/2609.34346/) (2026)
- [FracGen: Learning How Objects Stretch and Tear with Physics-Informed Video Generation](https://mlsanity.com/paper/2609.38152/) (2026)
- [Generative Cinematographer: Composing Camera and Object Motion in 3D](https://mlsanity.com/paper/2610.02180/) (2026)
- [Beyond Successful Robot Demonstrations: Learning from Human Interactions and Modeling Failure Dynamics](https://mlsanity.com/paper/2610.13522/) (2026)
- [When Left Is Right? Camera-Motion Understanding beyond Screen](https://mlsanity.com/paper/2610.17242/) (2026)

## Abstract (arXiv)

Dense point tracking is a fundamental problem in computer vision, with applications ranging from video analysis to robotic manipulation. State-of-the-art trackers typically rely on cost volumes to match features across frames, but this approach incurs quadratic complexity in spatial resolution, limiting scalability and efficiency. In this paper, we propose method, a novel dense point tracker that eschews cost volumes in favor of warping. Inspired by recent advances in optical flow, our approach iteratively refines track estimates by warping features from the target frame to the query frame based on the current estimate. Combined with a transformer architecture that performs joint spatiotemporal reasoning across all tracks, our design establishes long-range correspondences without computing feature correlations. Our model is simple and achieves state-of-the-art performance on standard dense point tracking benchmarks, including TAP-Vid-DAVIS, TAP-Vid-Kinetics, and Robo-TAP. Remarkably, the model also excels at optical flow, sometimes outperforming specialized methods on the Sintel, KITTI, and Spring benchmarks. These results suggest that warping-based architectures can unify dense point tracking and optical flow estimation.