# MVTrack: Ultrafast Appearance-Free Moving Object Tracking from Compressed Bitstreams

arXiv: https://arxiv.org/abs/2608.10790 · HTML: https://mlsanity.com/paper/2608.10790/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 26 of its 49 references that are in the map.
- First posted on arXiv 11 Aug 2026.

## Builds on (most cited first, 26 of 26)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Optuna: A Next-generation Hyperparameter Optimization Framework](https://mlsanity.com/paper/1907.10902/) (2019)
- [Objects as Points](https://mlsanity.com/paper/1904.07850/) (2019)
- [ByteTrack: Multi-Object Tracking by Associating Every Detection Box](https://mlsanity.com/paper/2110.06864/) (2021)
- [Performance Measures and a Data Set for Multi-Target, Multi-Camera Tracking](https://mlsanity.com/paper/1609.01775/) (2016)
- [HOTA: A Higher Order Metric for Evaluating Multi-Object Tracking](https://mlsanity.com/paper/2009.07736/) (2020)
- [Delving Deeper into Convolutional Networks for Learning Video Representations](https://mlsanity.com/paper/1511.06432/) (2015)
- [Compressed Video Action Recognition](https://mlsanity.com/paper/1712.00636/) (2017)
- [Enhancing Geometric Factors in Model Learning and Inference for Object Detection and Instance Segmentation](https://mlsanity.com/paper/2005.03572/) (2020)
- [Real-time Action Recognition with Enhanced Motion Vector CNNs](https://mlsanity.com/paper/1604.07669/) (2016)
- [Fast Object Detection in Compressed Video](https://mlsanity.com/paper/1811.11057/) (2018)
- [Efficient Motion-Aware Video MLLM](https://mlsanity.com/paper/2503.13016/) (2025)
- [CoPE-VideoLM: Leveraging Codec Primitives For Efficient Video Language Modeling](https://mlsanity.com/paper/2602.13191/) (2026)
- [Accelerating Video Object Segmentation with Compressed Video](https://mlsanity.com/paper/2107.12192/) (2021)
- [Efficient Semantic Segmentation by Altering Resolutions for Compressed Videos](https://mlsanity.com/paper/2303.07224/) (2023)
- [MVFlow: Deep Optical Flow Estimation of Compressed Videos with Motion Vector Prior](https://mlsanity.com/paper/2308.01568/) (2023)
- [MV-YOLO: Motion Vector-aided Tracking by Semantic Object Detection](https://mlsanity.com/paper/1805.00107/) (2018)
- [Real-time Online Multi-Object Tracking in Compressed Domain](https://mlsanity.com/paper/2204.02081/) (2022)
- [CoVA: Exploiting Compressed-Domain Analysis to Accelerate Video Analytics](https://mlsanity.com/paper/2207.00588/) (2022)
- [Motion Vector Extrapolation for Video Object Detection](https://mlsanity.com/paper/2104.08918/) (2021)
- [See Without Decoding: Motion-Vector-Based Tracking in Compressed Video](https://mlsanity.com/paper/2602.00153/) (2026)
- [Can you tell a face from a HEVC bitstream?](https://mlsanity.com/paper/1709.02993/) (2017)
- [Efficient Temporally-Aware DeepFake Detection using H.264 Motion Vectors](https://mlsanity.com/paper/2311.10788/) (2023)
- [MoCLIP-Lite: Efficient Video Recognition by Fusing CLIP with Motion Vectors](https://mlsanity.com/paper/2509.17084/) (2025)
- [MVP: Motion Vector Propagation for Zero-Shot Video Object Detection](https://mlsanity.com/paper/2509.18388/) (2025)

## Abstract (arXiv)

Deploying modern video trackers at scale is bottlenecked by the computational cost of RGB-based object detectors. To this end, we present MVTrack, an ultrafast tracker for moving objects that operates directly on H.264 bitstreams. MVTrack combines MVDet, a lightweight detector for motion vector fields, with MVLink, a minimalist kinematic association module. On VIRAT, MVTrack outperforms YOLO26n while using 60× fewer parameters, requiring 40× fewer FLOPs, and reducing CPU latency by 8.6×. These results demonstrate that compressed video data alone can enable accurate and scalable surveillance tracking, thereby bypassing the need for pixel reconstruction.