# 1st Place Solution for CVPR2023 BURST Long Tail and Open World Challenges

arXiv: https://arxiv.org/abs/2308.04598 · HTML: https://mlsanity.com/paper/2308.04598/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 18 of its 19 references that are in the map.
- First posted on arXiv 8 Aug 2023.

## Builds on (most cited first, 18 of 18)

- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [Simple Online and Realtime Tracking](https://mlsanity.com/paper/1602.00763/) (2016)
- [ByteTrack: Multi-Object Tracking by Associating Every Detection Box](https://mlsanity.com/paper/2110.06864/) (2021)
- [FairMOT: On the Fairness of Detection and Re-Identification in Multiple Object Tracking](https://mlsanity.com/paper/2004.01888/) (2020)
- [MOTR: End-to-End Multiple-Object Tracking with Transformer](https://mlsanity.com/paper/2105.03247/) (2021)
- [TransTrack: Multiple Object Tracking with Transformer](https://mlsanity.com/paper/2012.15460/) (2020)
- [MOTS: Multi-Object Tracking and Segmentation](https://mlsanity.com/paper/1902.03604/) (2019)
- [Quasi-Dense Similarity Learning for Multiple Object Tracking](https://mlsanity.com/paper/2006.06664/) (2020)
- [Track to Detect and Segment: An Online Multi-Object Tracker](https://mlsanity.com/paper/2103.08808/) (2021)
- [Rethinking the competition between detection and ReID in Multi-Object Tracking](https://mlsanity.com/paper/2010.12138/) (2020)
- [Towards Grand Unification of Object Tracking](https://mlsanity.com/paper/2207.07078/) (2022)
- [CBNet: A Composite Backbone Network Architecture for Object Detection](https://mlsanity.com/paper/2107.00420/) (2021)
- [RetinaTrack: Online Single Stage Joint Detection and Tracking](https://mlsanity.com/paper/2003.13870/) (2020)
- [Do Different Tracking Tasks Require Different Appearance Models?](https://mlsanity.com/paper/2107.02156/) (2021)
- [Tracking Every Thing in the Wild](https://mlsanity.com/paper/2207.12978/) (2022)
- [Prototypical Cross-Attention Networks for Multiple Object Tracking and Segmentation](https://mlsanity.com/paper/2106.11958/) (2021)

## Abstract (arXiv)

Currently, Video Instance Segmentation (VIS) aims at segmenting and categorizing objects in videos from a closed set of training categories that contain only a few dozen of categories, lacking the ability to handle diverse objects in real-world videos. As TAO and BURST datasets release, we have the opportunity to research VIS in long-tailed and open-world scenarios. Traditional VIS methods are evaluated on benchmarks limited to a small number of common classes, But practical applications require trackers that go beyond these common classes, detecting and tracking rare and even never-before-seen objects. Inspired by the latest MOT paper for the long tail task (Tracking Every Thing in the Wild, Siyuan Li et), for the BURST long tail challenge, we train our model on a combination of LVISv0.5 and the COCO dataset using repeat factor sampling. First, train the detector with segmentation and CEM on LVISv0.5 + COCO dataset. And then, train the instance appearance similarity head on the TAO dataset. at last, our method (LeTracker) gets 14.9 HOTAall in the BURST test set, ranking 1st in the benchmark. for the open-world challenges, we only use 64 classes (Intersection classes of BURST Train subset and COCO dataset, without LVIS dataset) annotations data training, and testing on BURST test set data and get 61.4 OWTAall, ranking 1st in the benchmark. Our code will be released to facilitate future research.