mlsanityData to 14 Oct 2026

The generator is the tracker: Multi-object tracking by painting persistent identity colours

arXiv 2607.17120 ↗First posted 19 Jul 2026cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Multi-object tracking (MOT) is conventionally decomposed into detection followed by association, with object identity maintained as external state: track buffers, motion models, and appearance embeddings. We ask whether a video generator can maintain that state in pixels. We fine-tune a 22B text-to-video diffusion model (LTX-2.3) with a lightweight in-context LoRA to translate an RGB clip into an ID-map clip, a video in which every person is painted a flat, distinct color that persists over time: same color, same identity. Long videos are generated as chained windows, where each window is conditioned on the cleaned tail of the previous one. A brief continuation fine-tune teaches the model to extend a given coloring, after which identity flows through the chain with no tracker, no motion model, and no re-identification module. On the DanceTrack test server, our system, to our knowledge the first generative tracker evaluated there and the only entry with no detector and no tracking stack, reaches 40.3 HOTA. This is well below today's specialist state of the art (>=70 HOTA), but with a unique, inverted error profile: its association score (AssA 44.1) exceeds every tracker of the original benchmark suite while detection remains the sole deficit. Controlled comparisons show the mechanism matters: the same generated windows linked by classical post-hoc association score 2x worse (18.2 HOTA), and frame-to-frame IoU association fragments tracks that the generator's colors keep whole. On 383 mined occlusion events, the generator re-acquires identities after gaps at a 42% conditional rate where appearance-embedding baselines score zero, including gaps longer than its temporal context, evidence that the generator's color assignment functions as an emergent re-identification signal. We release code, checkpoints, and the full pre-registered experimental log.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 21 of its 22 references that are in the map. First posted on arXiv 19 Jul 2026.

Builds on

21 of 22 references.

References · 21

  1. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  2. 2023Segment Anythingcited by 8,234
  3. 2022Flow Matching for Generative Modelingcited by 5,702
  4. 2022Video Diffusion Modelscited by 2,305
  5. 2017Simple Online and Realtime Tracking with a Deep Association Metriccited by 910
  6. 2016Simple Online and Realtime Trackingcited by 868
  7. 2021ByteTrack: Multi-Object Tracking by Associating Every Detection Boxcited by 823
  8. 2024LTX-Video: Realtime Video Latent Diffusioncited by 581
  9. 2020HOTA: A Higher Order Metric for Evaluating Multi-Object Trackingcited by 552
  10. 2020FairMOT: On the Fairness of Detection and Re-Identification in Multiple Object Trackingcited by 490
  11. 2021MOTR: End-to-End Multiple-Object Tracking with Transformercited by 381
  12. 2020TransTrack: Multiple Object Tracking with Transformercited by 371
9 more
  1. 2022Observation-Centric SORT: Rethinking SORT for Robust Multi-Object Trackingcited by 346
  2. 2022Images Speak in Images: A Generalist Painter for In-Context Visual Learningcited by 318
  3. 2021DanceTrack: Multi-Object Tracking in Uniform Appearance and Diverse Motioncited by 217
  4. 2022A Unified Sequence Interface for Vision Taskscited by 152
  5. 2022MOTRv2: Bootstrapping End-to-End Multi-Object Tracking by Pretrained Object Detectorscited by 148
  6. 2023Deep OC-SORT: Multi-Pedestrian Tracking by Adaptive Re-Identificationcited by 132
  7. 2023MeMOTR: Long-Term Memory-Augmented Transformer for Multi-Object Trackingcited by 88
  8. 2024DiffMOT: A Real-time Diffusion-based Multiple Object Tracker with Non-linear Predictioncited by 59
  9. 2026Image Generators are Generalist Vision Learnerscited by 43

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Track4Gen: Teaching Video Diffusion Models to Track Points Improves Video Generation
  2. 2025Video Diffusion Models Excel at Tracking Similar-Looking Objects Without Supervision
  3. 2025Point Prompting: Counterfactual Tracking with Video Diffusion Models
  4. 2025Generative Point Tracking with Flow Matching
  5. 2025MagicMirror: ID-Preserved Video Generation in Video Diffusion Transformers
  6. 2025Emergent Temporal Correspondences from Video Diffusion Transformers
  7. 2026TRACE: Trajectory Representation and Consistency Estimation for AI-Generated Video Detection
  8. 2023DreaMoving: A Human Video Generation Framework based on Diffusion Models