# Multi-resolution Rescored ByteTrack for Video Object Detection on Ultra-low-power Embedded Systems

arXiv: https://arxiv.org/abs/2404.11488 · HTML: https://mlsanity.com/paper/2404.11488/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 34 references that are in the map.
- First posted on arXiv 17 Apr 2024.

## Builds on (most cited first, 23 of 23)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [YOLOv3: An Incremental Improvement](https://mlsanity.com/paper/1804.02767/) (2018)
- [YOLOX: Exceeding YOLO Series in 2021](https://mlsanity.com/paper/2107.08430/) (2021)
- [EfficientDet: Scalable and Efficient Object Detection](https://mlsanity.com/paper/1911.09070/) (2019)
- [ByteTrack: Multi-Object Tracking by Associating Every Detection Box](https://mlsanity.com/paper/2110.06864/) (2021)
- [Flow-Guided Feature Aggregation for Video Object Detection](https://mlsanity.com/paper/1703.10025/) (2017)
- [Memory Enhanced Global-Local Aggregation for Video Object Detection](https://mlsanity.com/paper/2003.12063/) (2020)
- [Seq-NMS for Video Object Detection](https://mlsanity.com/paper/1602.08465/) (2016)
- [MobileDets: Searching for Object Detection Architectures for Mobile Accelerators](https://mlsanity.com/paper/2004.14525/) (2020)
- [End-to-End Video Object Detection with Spatial-Temporal Transformers](https://mlsanity.com/paper/2105.10920/) (2021)
- [Dynamic Resolution Network](https://mlsanity.com/paper/2106.02898/) (2021)
- [Looking Fast and Slow: Memory-Guided Mobile Video Object Detection](https://mlsanity.com/paper/1903.10172/) (2019)
- [Vega: A 10-Core SoC for IoT End-Nodes with DNN Acceleration and Cognitive Wake-Up From MRAM-Based State-Retentive Sleep Mode](https://mlsanity.com/paper/2110.09101/) (2021)
- [YOLOV: Making Still Image Object Detectors Great at Video Object Detection](https://mlsanity.com/paper/2208.09686/) (2022)
- [Integrated Object Detection and Tracking with Tracklet-Conditioned Detection](https://mlsanity.com/paper/1811.11167/) (2018)
- [Bio-inspired Autonomous Exploration Policies with CNN-based Object Detection on Nano-drones](https://mlsanity.com/paper/2301.12175/) (2023)
- [BlockCopy: High-Resolution Video Processing with Block-Sparse Feature Propagation and Online Policies](https://mlsanity.com/paper/2108.09376/) (2021)
- [Adaptive Deep Learning for Efficient Visual Pose Estimation aboard Ultra-low-power Nano-drones](https://mlsanity.com/paper/2401.15236/) (2024)

## Built on by (most cited first, 4 of 4)

- [TiROD: Tiny Robotics Dataset and Benchmark for Continual Object Detection](https://mlsanity.com/paper/2409.16215/) (2024)
- [MR2-ByteTrack: CNN and Transformer-based Video Object Detection for AI-augmented Embedded Vision Sensor Nodes](https://mlsanity.com/paper/2605.15423/) (2026)
- [Embedded Arena: Iterative Optimization via Hardware Feedback](https://mlsanity.com/paper/2606.16190/) (2026)
- [Adaptive AI: Energy Efficient Multi-exit TinyML on Intelligent Vision Systems at the Edge](https://mlsanity.com/paper/2609.11939/) (2026)

## Abstract (arXiv)

This paper introduces Multi-Resolution Rescored Byte-Track (MR2-ByteTrack), a novel video object detection framework for ultra-low-power embedded processors. This method reduces the average compute load of an off-the-shelf Deep Neural Network (DNN) based object detector by up to 2.25× by alternating the processing of high-resolution images (320×320 pixels) with multiple down-sized frames (192×192 pixels). To tackle the accuracy degradation due to the reduced image input size, MR2-ByteTrack correlates the output detections over time using the ByteTrack tracker and corrects potential misclassification using a novel probabilistic Rescore algorithm. By interleaving two down-sized images for every high-resolution one as the input of different state-of-the-art DNN object detectors with our MR2-ByteTrack, we demonstrate an average accuracy increase of 2.16% and a latency reduction of 43% on the GAP9 microcontroller compared to a baseline frame-by-frame inference scheme using exclusively full-resolution images. Code available at: https://github.com/Bomps4/MultiResolutionRescoredByteTrack