mlsanityData to 14 Oct 2026

Data-Model-Circuit Tri-Design for Ultra-Light Video Intelligence on Edge Devices

arXiv 2210.08578 ↗First posted 16 Oct 2022cs.CV cs.ARcited by 4

Not on the map. Why?

Abstract (arXiv)

In this paper, we propose a data-model-hardware tri-design framework for high-throughput, low-cost, and high-accuracy multi-object tracking (MOT) on High-Definition (HD) video stream. First, to enable ultra-light video intelligence, we propose temporal frame-filtering and spatial saliency-focusing approaches to reduce the complexity of massive video data. Second, we exploit structure-aware weight sparsity to design a hardware-friendly model compression method. Third, assisted with data and model complexity reduction, we propose a sparsity-aware, scalable, and low-power accelerator design, aiming to deliver real-time performance with high energy efficiency. Different from existing works, we make a solid step towards the synergized software/hardware co-optimization for realistic MOT model implementation. Compared to the state-of-the-art MOT baseline, our tri-design approach can achieve 12.5x latency reduction, 20.9x effective frame rate improvement, 5.83x lower power, and 9.78x better energy efficiency, without much accuracy drop.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 20 of its 27 references that are in the map. First posted on arXiv 16 Oct 2022.

Builds on

20 of 27 references.

References · 20

  1. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
  2. 2016Feature Pyramid Networks for Object Detectioncited by 5,036
  3. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
  4. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,262
  5. 2015Learning both Weights and Connections for Efficient Neural Networkscited by 2,031
  6. 2018Unsupervised Feature Learning via Non-Parametric Instance-level Discriminationcited by 1,845
  7. 2018BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learningcited by 1,288
  8. 2019A Survey of Autonomous Driving: Common Practices and Emerging Technologiescited by 379
  9. 2019Towards Real-Time Multi-Object Trackingcited by 311
  10. 2020Quasi-Dense Similarity Learning for Multiple Object Trackingcited by 220
  11. 2015Structured Pruning of Deep Convolutional Neural Networkscited by 149
  12. 2017Deep Reinforcement Learning for Unsupervised Video Summarization with Diversity-Representativeness Rewardcited by 105
8 more
  1. 2020PatDNN: Achieving Real-Time DNN Execution on Mobile Devices with Pattern-based Weight Pruningcited by 76
  2. 2018The History Began from AlexNet: A Comprehensive Survey on Deep Learning Approachescited by 75
  3. 2020Tracking by Instance Detection: A Meta-Learning Approachcited by 61
  4. 2019FPGA/DNN Co-Design: An Efficient Design Methodology for IoT Intelligence on the Edgecited by 48
  5. 2021Fast convolutional neural networks on FPGAs with hls4mlcited by 42
  6. 2020An Image Enhancing Pattern-based Sparsity for Real-time Inference on Mobile Devicescited by 18
  7. 2021Deployment of Deep Neural Networks for Object Detection on Edge AI Devices with Runtime Optimizationcited by 8
  8. 2021Compressed Object Detectioncited by 1

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 2.

4 papers.

  1. 2023Model Sparsity Can Simplify Machine Unlearningcited by 214
  2. 2024Defensive Unlearning with Adversarial Training for Robust Concept Erasure in Diffusion Modelscited by 168
  3. 2023Text-Visual Prompting for Efficient 2D Temporal Video Groundingcited by 18
  4. 2024The Power of Few: Accelerating and Enhancing Data Reweighting with Coreset Selectioncited by 3