# Track-On: Transformer-based Online Point Tracking with Memory

arXiv: https://arxiv.org/abs/2501.18487 · HTML: https://mlsanity.com/paper/2501.18487/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Jan 2025.

## Built on by (most cited first, 21 of 21)

- [TAPNext: Tracking Any Point (TAP) as Next Token Prediction](https://mlsanity.com/paper/2504.05579/) (2025)
- [TAPNext++: What's Next for Tracking Any Point (TAP)?](https://mlsanity.com/paper/2604.10582/) (2026)
- [LiteTracker: Leveraging Temporal Causality for Accurate Low-latency Tissue Tracking](https://mlsanity.com/paper/2504.09904/) (2025)
- [Video Understanding: From Geometry and Semantics to Unified Models](https://mlsanity.com/paper/2603.17840/) (2026)
- [MegaFlow: Zero-Shot Large Displacement Optical Flow](https://mlsanity.com/paper/2603.25739/) (2026)
- [Look Around and Pay Attention: Multi-camera Point Tracking Reimagined with Transformers](https://mlsanity.com/paper/2512.04213/) (2025)
- [Online Dense Point Tracking with Streaming Memory](https://mlsanity.com/paper/2503.06471/) (2025)
- [AnthroTAP: Learning Point Tracking with Real-World Motion](https://mlsanity.com/paper/2507.06233/) (2025)
- [Track-On2: Enhancing Online Point Tracking with Memory](https://mlsanity.com/paper/2509.19115/) (2025)
- [Probing and Leveraging Video Diffusion Transformer Features for Robust Point Tracking](https://mlsanity.com/paper/2512.20606/) (2025)
- [Taming Modern Point Tracking for Speckle Tracking Echocardiography via Impartial Motion](https://mlsanity.com/paper/2507.10127/) (2025)
- [BleedOrigin: Dynamic Bleeding Source Localization in Endoscopic Submucosal Dissection via Dual-Stage Detection and Tracking](https://mlsanity.com/paper/2507.15094/) (2025)
- [Is This Tracker On? A Benchmark Protocol for Dynamic Tracking](https://mlsanity.com/paper/2510.19819/) (2025)
- [SAM 2++: Tracking Anything at Any Granularity](https://mlsanity.com/paper/2510.18822/) (2025)
- [TAPVid-MV: A Benchmark for Tracking Any Point in 3D Across Multiple Views](https://mlsanity.com/paper/2609.01899/) (2026)
- [You Are Your Best Teacher: Semi-Supervised Surgical Point Tracking with Cycle-Consistent Self-Distillation](https://mlsanity.com/paper/2505.05722/) (2025)
- [Lattice Boltzmann Model for Learning Real-World Pixel Dynamicity](https://mlsanity.com/paper/2509.16527/) (2025)
- [K-Track: Kalman-Enhanced Tracking for Accelerating Deep Point Trackers on Edge Devices](https://mlsanity.com/paper/2512.10628/) (2025)
- [Real-World Point Tracking with Verifier-Guided Pseudo-Labeling](https://mlsanity.com/paper/2603.12217/) (2026)
- [M2P: Improving Visual Foundation Models with Mask-to-Point Weakly-Supervised Learning for Dense Point Tracking](https://mlsanity.com/paper/2603.17813/) (2026)
- [ReWorld-Track: A Recursive Event World Model for Language-Guided Multi-Camera Tracking](https://mlsanity.com/paper/2609.36677/) (2026)

## Abstract (arXiv)

In this paper, we consider the problem of long-term point tracking, which requires consistent identification of points across multiple frames in a video, despite changes in appearance, lighting, perspective, and occlusions. We target online tracking on a frame-by-frame basis, making it suitable for real-world, streaming scenarios. Specifically, we introduce Track-On, a simple transformer-based model designed for online long-term point tracking. Unlike prior methods that depend on full temporal modeling, our model processes video frames causally without access to future frames, leveraging two memory modules -- spatial memory and context memory -- to capture temporal information and maintain reliable point tracking over long time horizons. At inference time, it employs patch classification and refinement to identify correspondences and track points with high accuracy. Through extensive experiments, we demonstrate that Track-On sets a new state-of-the-art for online models and delivers superior or competitive results compared to offline approaches on seven datasets, including the TAP-Vid benchmark. Our method offers a robust and scalable solution for real-time tracking in diverse applications. Project page: https://kuis-ai.github.io/track_on