# ZJU ReLER Submission for EPIC-KITCHEN Challenge 2023: TREK-150 Single Object Tracking

arXiv: https://arxiv.org/abs/2307.02508 · HTML: https://mlsanity.com/paper/2307.02508/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 32 references that are in the map.
- First posted on arXiv 5 Jul 2023.

## Builds on (most cited first, 25 of 25)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [The 2017 DAVIS Challenge on Video Object Segmentation](https://mlsanity.com/paper/1704.00675/) (2017)
- [GOT-10k: A Large High-Diversity Benchmark for Generic Object Tracking in the Wild](https://mlsanity.com/paper/1810.11981/) (2018)
- [LaSOT: A High-quality Benchmark for Large-scale Single Object Tracking](https://mlsanity.com/paper/1809.07845/) (2018)
- [YouTube-VOS: A Large-Scale Video Object Segmentation Benchmark](https://mlsanity.com/paper/1809.03327/) (2018)
- [MixFormer: End-to-End Tracking with Iterative Mixed Attention](https://mlsanity.com/paper/2203.11082/) (2022)
- [MOSE: A New Dataset for Video Object Segmentation in Complex Scenes](https://mlsanity.com/paper/2302.01872/) (2023)
- [Associating Objects with Transformers for Video Object Segmentation](https://mlsanity.com/paper/2106.02638/) (2021)
- [Segment and Track Anything](https://mlsanity.com/paper/2305.06558/) (2023)
- [Decoupling Features in Hierarchical Propagation for Video Object Segmentation](https://mlsanity.com/paper/2210.09782/) (2022)
- [Collaborative Video Object Segmentation by Foreground-Background Integration](https://mlsanity.com/paper/2003.08333/) (2020)
- [EPIC-KITCHENS VISOR Benchmark: VIdeo Segmentations and Object Relations](https://mlsanity.com/paper/2209.13064/) (2022)
- [A Survey on Deep Learning Technique for Video Segmentation](https://mlsanity.com/paper/2107.01153/) (2021)
- [SwinTrack: A Simple and Strong Baseline for Transformer Tracking](https://mlsanity.com/paper/2112.00995/) (2021)
- [Alpha-Refine: Boosting Tracking Performance by Precise Bounding Box Estimation](https://mlsanity.com/paper/2007.02024/) (2020)
- [Collaborative Video Object Segmentation by Multi-Scale Foreground-Background Integration](https://mlsanity.com/paper/2010.06349/) (2020)
- [Robust Visual Tracking by Segmentation](https://mlsanity.com/paper/2203.11191/) (2022)
- [EgoTracks: A Long-term Egocentric Visual Object Tracking Dataset](https://mlsanity.com/paper/2301.03213/) (2023)
- [Locality-Aware Inter-and Intra-Video Reconstruction for Self-Supervised Correspondence Learning](https://mlsanity.com/paper/2203.14333/) (2022)
- [Visual Object Tracking in First Person Vision](https://mlsanity.com/paper/2209.13502/) (2022)
- [Is First Person Vision Challenging for Object Tracking?](https://mlsanity.com/paper/2108.13665/) (2021)
- [Unified Sequence-to-Sequence Learning for Single- and Multi-Modal Visual Object Tracking](https://mlsanity.com/paper/2304.14394/) (2023)
- [Unified Mask Embedding and Correspondence Learning for Self-Supervised Video Segmentation](https://mlsanity.com/paper/2303.10100/) (2023)

## Built on by (most cited first, 1 of 1)

- [Segment Anything for Videos: A Systematic Survey](https://mlsanity.com/paper/2408.08315/) (2024)

## Abstract (arXiv)

The Associating Objects with Transformers (AOT) framework has exhibited exceptional performance in a wide range of complex scenarios for video object tracking and segmentation. In this study, we convert the bounding boxes to masks in reference frames with the help of the Segment Anything Model (SAM) and Alpha-Refine, and then propagate the masks to the current frame, transforming the task from Video Object Tracking (VOT) to video object segmentation (VOS). Furthermore, we introduce MSDeAOT, a variant of the AOT series that incorporates transformers at multiple feature scales. MSDeAOT efficiently propagates object masks from previous frames to the current frame using two feature scales of 16 and 8. As a testament to the effectiveness of our design, we achieved the 1st place in the EPIC-KITCHENS TREK-150 Object Tracking Challenge.