# SkyNet: a Hardware-Efficient Method for Object Detection and Tracking on Embedded Systems

arXiv: https://arxiv.org/abs/1909.09709 · HTML: https://mlsanity.com/paper/1909.09709/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Sep 2019.

## Built on by (most cited first, 27 of 27)

- [GPT4AIGChip: Towards Next-Generation AI Accelerator Design Automation via Large Language Models](https://mlsanity.com/paper/2309.10730/) (2023)
- [NAAS: Neural Accelerator Architecture Search](https://mlsanity.com/paper/2105.13258/) (2021)
- [Generic Neural Architecture Search via Regression](https://mlsanity.com/paper/2108.01899/) (2021)
- [Object Detection in Autonomous Vehicles: Status and Open Challenges](https://mlsanity.com/paper/2201.07706/) (2022)
- [A Survey on Approximate Edge AI for Energy Efficient Autonomous Driving Services](https://mlsanity.com/paper/2304.14271/) (2023)
- [Enabling Design Methodologies and Future Trends for Edge AI: Specialization and Co-design](https://mlsanity.com/paper/2103.15750/) (2021)
- [YOLO-ReT: Towards High Accuracy Real-time Object Detection on Edge GPUs](https://mlsanity.com/paper/2110.13713/) (2021)
- [HLSFactory: A Framework Empowering High-Level Synthesis Datasets for Machine Learning and Beyond](https://mlsanity.com/paper/2405.00820/) (2024)
- [HiKonv: High Throughput Quantized Convolution With Novel Bit-wise Management and Computation](https://mlsanity.com/paper/2112.13972/) (2021)
- [WinoCNN: Kernel Sharing Winograd Systolic Array for Efficient Convolutional Neural Network Acceleration on FPGAs](https://mlsanity.com/paper/2107.04244/) (2021)
- [Efficient Deep Learning Infrastructures for Embedded Computing Systems: A Comprehensive Survey and Future Envision](https://mlsanity.com/paper/2411.01431/) (2024)
- [EH-DNAS: End-to-End Hardware-aware Differentiable Neural Architecture Search](https://mlsanity.com/paper/2111.12299/) (2021)
- [Reaching the Edge of the Edge: Image Analysis in Space](https://mlsanity.com/paper/2301.04954/) (2023)
- [E3-UAV: An Edge-based Energy-Efficient Object Detection System for Unmanned Aerial Vehicles](https://mlsanity.com/paper/2308.04774/) (2023)
- [Pseudo-IoU: Improving Label Assignment in Anchor-Free Object Detection](https://mlsanity.com/paper/2104.14082/) (2021)
- [Visual Domain Adaptation for Monocular Depth Estimation on Resource-Constrained Hardware](https://mlsanity.com/paper/2108.02671/) (2021)
- [DeepBurning-MixQ: An Open Source Mixed-Precision Neural Network Accelerator Design Framework for FPGAs](https://mlsanity.com/paper/2308.11334/) (2023)
- [You Only Search Once: On Lightweight Differentiable Architecture Search for Resource-Constrained Embedded Platforms](https://mlsanity.com/paper/2208.14446/) (2022)
- [Anomaly Detection in Residential Video Surveillance on Edge Devices in IoT Framework](https://mlsanity.com/paper/2107.04767/) (2021)
- [You Better Look Twice: a new perspective for designing accurate detectors with reduced computations](https://mlsanity.com/paper/2107.10050/) (2021)
- [Streaming Video Analytics On The Edge With Asynchronous Cloud Support](https://mlsanity.com/paper/2210.01402/) (2022)
- [AI+HW 2035: Shaping the Next Decade](https://mlsanity.com/paper/2603.05225/) (2026)
- [RSCA: Real-time Segmentation-based Context-Aware Scene Text Detection](https://mlsanity.com/paper/2105.12789/) (2021)
- [Adaptive Subsampling for ROI-based Visual Tracking: Algorithms and FPGA Implementation](https://mlsanity.com/paper/2112.09775/) (2021)
- [Compilation and Optimizations for Efficient Machine Learning on Embedded Systems](https://mlsanity.com/paper/2206.03326/) (2022)
- [HiKonv: Maximizing the Throughput of Quantized Convolution With Novel Bit-wise Management and Computation](https://mlsanity.com/paper/2208.00763/) (2022)
- [A3C3: AI Algorithm and Accelerator Co-design, Co-search, and Co-generation](https://mlsanity.com/paper/2606.20869/) (2026)

## Abstract (arXiv)

Object detection and tracking are challenging tasks for resource-constrained embedded systems. While these tasks are among the most compute-intensive tasks from the artificial intelligence domain, they are only allowed to use limited computation and memory resources on embedded devices. In the meanwhile, such resource-constrained implementations are often required to satisfy additional demanding requirements such as real-time response, high-throughput performance, and reliable inference accuracy. To overcome these challenges, we propose SkyNet, a hardware-efficient neural network to deliver the state-of-the-art detection accuracy and speed for embedded systems. Instead of following the common top-down flow for compact DNN (Deep Neural Network) design, SkyNet provides a bottom-up DNN design approach with comprehensive understanding of the hardware constraints at the very beginning to deliver hardware-efficient DNNs. The effectiveness of SkyNet is demonstrated by winning the competitive System Design Contest for low power object detection in the 56th IEEE/ACM Design Automation Conference (DAC-SDC), where our SkyNet significantly outperforms all other 100+ competitors: it delivers 0.731 Intersection over Union (IoU) and 67.33 frames per second (FPS) on a TX2 embedded GPU; and 0.716 IoU and 25.05 FPS on an Ultra96 embedded FPGA. The evaluation of SkyNet is also extended to GOT-10K, a recent large-scale high-diversity benchmark for generic object tracking in the wild. For state-of-the-art object trackers SiamRPN++ and SiamMask, where ResNet-50 is employed as the backbone, implementations using our SkyNet as the backbone DNN are 1.60X and 1.73X faster with better or similar accuracy when running on a 1080Ti GPU, and 37.20X smaller in terms of parameter size for significantly better memory and storage footprint.