# AllTracker: Efficient Dense Point Tracking at High Resolution

arXiv: https://arxiv.org/abs/2506.07310 · HTML: https://mlsanity.com/paper/2506.07310/

## Key facts

- Cited by 58 papers in the map (data through 14 Oct 2026).
- Builds on 42 of its 63 references that are in the map.
- First posted on arXiv 8 Jun 2025.

## Builds on (most cited first, 30 of 42)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [A ConvNet for the 2020s](https://mlsanity.com/paper/2201.03545/) (2022)
- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [MLP-Mixer: An all-MLP Architecture for Vision](https://mlsanity.com/paper/2105.01601/) (2021)
- [Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network](https://mlsanity.com/paper/1609.05158/) (2016)
- [FlowNet: Learning Optical Flow with Convolutional Networks](https://mlsanity.com/paper/1504.06852/) (2015)
- [PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume](https://mlsanity.com/paper/1709.02371/) (2017)
- [A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimation](https://mlsanity.com/paper/1512.02134/) (2015)
- [FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks](https://mlsanity.com/paper/1612.01925/) (2016)
- [CoTracker: It is Better to Track Together](https://mlsanity.com/paper/2307.07635/) (2023)
- [TartanAir: A Dataset to Push the Limits of Visual SLAM](https://mlsanity.com/paper/2003.14338/) (2020)
- [Kubric: A scalable dataset generator](https://mlsanity.com/paper/2203.03570/) (2022)
- [GMFlow: Learning Optical Flow via Global Matching](https://mlsanity.com/paper/2111.13680/) (2021)
- [TAPIR: Tracking Any Point with per-frame Initialization and temporal Refinement](https://mlsanity.com/paper/2306.08637/) (2023)
- [PointOdyssey: A Large-Scale Synthetic Dataset for Long-Term Point Tracking](https://mlsanity.com/paper/2307.15055/) (2023)
- [TAP-Vid: A Benchmark for Tracking Any Point in a Video](https://mlsanity.com/paper/2211.03726/) (2022)
- [Particle Video Revisited: Tracking Through Occlusions Using Point Trajectories](https://mlsanity.com/paper/2204.04153/) (2022)
- [SpatialTracker: Tracking Any 2D Pixels in 3D Space](https://mlsanity.com/paper/2404.04319/) (2024)
- [SEA-RAFT: Simple, Efficient, Accurate RAFT for Optical Flow](https://mlsanity.com/paper/2405.14793/) (2024)
- [Spring: A High-Resolution High-Detail Dataset and Benchmark for Scene Flow, Optical Flow and Stereo](https://mlsanity.com/paper/2303.01943/) (2023)
- [Playing for Benchmarks](https://mlsanity.com/paper/1709.07322/) (2017)
- [DynamicStereo: Consistent Dynamic Depth from Stereo Videos](https://mlsanity.com/paper/2305.02296/) (2023)
- [BootsTAP: Bootstrapped Training for Tracking-Any-Point](https://mlsanity.com/paper/2402.00847/) (2024)
- [AutoFlow: Learning a Better Training Set for Optical Flow](https://mlsanity.com/paper/2104.14544/) (2021)
- [RoboTAP: Tracking Arbitrary Points for Few-Shot Visual Imitation](https://mlsanity.com/paper/2308.15975/) (2023)
- [Creatures great and SMAL: Recovering the shape and motion of animals from video](https://mlsanity.com/paper/1811.05804/) (2018)
- [Local All-Pair Correspondence for Point Tracking](https://mlsanity.com/paper/2407.15420/) (2024)
- [Beyond Pick-and-Place: Tackling Robotic Stacking of Diverse Shapes](https://mlsanity.com/paper/2110.06192/) (2021)
- [Accurate Optical Flow via Direct Cost Volume Processing](https://mlsanity.com/paper/1704.07325/) (2017)

## Built on by (most cited first, 30 of 58)

- [Efficiently Reconstructing Dynamic Scenes One D4RT at a Time](https://mlsanity.com/paper/2512.08924/) (2025)
- [Rethinking Video Generation Model for the Embodied World](https://mlsanity.com/paper/2601.15282/) (2026)
- [Multi-View 3D Point Tracking](https://mlsanity.com/paper/2508.21060/) (2025)
- [Walk through Paintings: Egocentric World Models from Internet Priors](https://mlsanity.com/paper/2601.15284/) (2026)
- [CoWTracker: Tracking by Warping instead of Correlation](https://mlsanity.com/paper/2602.04877/) (2026)
- [Track4World: Feedforward World-centric Dense 3D Tracking of All Pixels](https://mlsanity.com/paper/2603.02573/) (2026)
- [Turning Video Models into Generalist Robot Policies](https://mlsanity.com/paper/2605.27817/) (2026)
- [What Happens Next? Anticipating Future Motion by Generating Point Trajectories](https://mlsanity.com/paper/2509.21592/) (2025)
- [PhysisForcing: Physics Reinforced World Simulator for Robotic Manipulation](https://mlsanity.com/paper/2606.28128/) (2026)
- [AdaGaR: Adaptive Gabor Representation for Dynamic Scene Reconstruction](https://mlsanity.com/paper/2601.00796/) (2026)
- [FunRec: Reconstructing Functional 3D Scenes from Egocentric Interaction Videos](https://mlsanity.com/paper/2604.05621/) (2026)
- [Trokens: Semantic-Aware Relational Trajectory Tokens for Few-Shot Action Recognition](https://mlsanity.com/paper/2508.03695/) (2025)
- [AnimaMimic: Imitating 3D Animation from Video Priors](https://mlsanity.com/paper/2512.14133/) (2025)
- [TAPNext++: What's Next for Tracking Any Point (TAP)?](https://mlsanity.com/paper/2604.10582/) (2026)
- [ObjectForesight: Predicting Future 3D Object Trajectories from Human Videos](https://mlsanity.com/paper/2601.05237/) (2026)
- [Motion Attribution for Video Generation](https://mlsanity.com/paper/2601.08828/) (2026)
- [AVControl: Efficient Framework for Training Audio-Visual Controls](https://mlsanity.com/paper/2603.24793/) (2026)
- [MegaFlow: Zero-Shot Large Displacement Optical Flow](https://mlsanity.com/paper/2603.25739/) (2026)
- [TrackCraft3R: Repurposing Video Diffusion Transformers for Dense 3D Tracking](https://mlsanity.com/paper/2605.12587/) (2026)
- [MolmoMotion: Forecasting Point Trajectories in 3D with Language Instruction](https://mlsanity.com/paper/2606.18558/) (2026)
- [MV-TAP: Tracking Any Point in Multi-View Videos](https://mlsanity.com/paper/2512.02006/) (2025)
- [TrajVG: 3D Trajectory-Coupled Visual Geometry Learning](https://mlsanity.com/paper/2602.04439/) (2026)
- [Probing and Leveraging Video Diffusion Transformer Features for Robust Point Tracking](https://mlsanity.com/paper/2512.20606/) (2025)
- [FreeArtGS: Articulated Gaussian Splatting Under Free-moving Scenario](https://mlsanity.com/paper/2603.22102/) (2026)
- [MoRight: Motion Control Done Right](https://mlsanity.com/paper/2604.07348/) (2026)
- [Reshoot-Anything: A Self-Supervised Model for In-the-Wild Video Reshooting](https://mlsanity.com/paper/2604.21776/) (2026)
- [VidMap: Exploiting Temporal Structure for Video-Based Structure-from-Motion](https://mlsanity.com/paper/2607.27194/) (2026)
- [PLOT: Pseudo-Labeling via Object Tracking for Monocular 3D Object Detection](https://mlsanity.com/paper/2507.02393/) (2025)
- [Generative Spatiotemporal Data Augmentation](https://mlsanity.com/paper/2512.12508/) (2025)
- [4D Primitive-Mach'e: Glueing Primitives for Persistent 4D Scene Reconstruction](https://mlsanity.com/paper/2512.16564/) (2025)

## Abstract (arXiv)

We introduce AllTracker: a model that estimates long-range point tracks by way of estimating the flow field between a query frame and every other frame of a video. Unlike existing point tracking methods, our approach delivers high-resolution and dense (all-pixel) correspondence fields, which can be visualized as flow maps. Unlike existing optical flow methods, our approach corresponds one frame to hundreds of subsequent frames, rather than just the next frame. We develop a new architecture for this task, blending techniques from existing work in optical flow and point tracking: the model performs iterative inference on low-resolution grids of correspondence estimates, propagating information spatially via 2D convolution layers, and propagating information temporally via pixel-aligned attention layers. The model is fast and parameter-efficient (16 million parameters), and delivers state-of-the-art point tracking accuracy at high resolution (i.e., tracking 768x1024 pixels, on a 40G GPU). A benefit of our design is that we can train jointly on optical flow datasets and point tracking datasets, and we find that doing so is crucial for top performance. We provide an extensive ablation study on our architecture details and training recipe, making it clear which details matter most. Our code and model weights are available at https://alltracker.github.io