# Mondrian: On-Device High-Performance Video Analytics with Compressive Packed Inference

arXiv: https://arxiv.org/abs/2403.07598 · HTML: https://mlsanity.com/paper/2403.07598/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 20 of its 68 references that are in the map.
- First posted on arXiv 12 Mar 2024.

## Builds on (most cited first, 20 of 20)

- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015)
- [YOLOv4: Optimal Speed and Accuracy of Object Detection](https://mlsanity.com/paper/2004.10934/) (2020)
- [EfficientDet: Scalable and Efficient Object Detection](https://mlsanity.com/paper/1911.09070/) (2019)
- [Towards Real-Time Multi-Object Tracking](https://mlsanity.com/paper/1909.12605/) (2019)
- [MOTChallenge: A Benchmark for Single-Camera Multiple Target Tracking](https://mlsanity.com/paper/2010.07548/) (2020)
- [SPINN: Synergistic Progressive Inference of Neural Networks over Device and Cloud](https://mlsanity.com/paper/2008.06402/) (2020)
- [SiamMOT: Siamese Multi-Object Tracking](https://mlsanity.com/paper/2105.11595/) (2021)
- [DeepCache: Principled Cache for Mobile Deep Vision](https://mlsanity.com/paper/1712.01670/) (2017)
- [Learning When and Where to Zoom with Deep Reinforcement Learning](https://mlsanity.com/paper/2003.00425/) (2020)
- [Dynamic Resolution Network](https://mlsanity.com/paper/2106.02898/) (2021)
- [The 5th AI City Challenge](https://mlsanity.com/paper/2104.12233/) (2021)
- [AdaScale: Towards Real-time Video Object Detection Using Adaptive Scaling](https://mlsanity.com/paper/1902.02910/) (2019)
- [ELASTIC: Improving CNNs with Dynamic Scaling Policies](https://mlsanity.com/paper/1812.05262/) (2018)
- [DynO: Dynamic Onloading of Deep Neural Networks from Cloud to Device](https://mlsanity.com/paper/2104.09949/) (2021)
- [ApproxDet: Content and Contention-Aware Approximate Object Detection for Mobiles](https://mlsanity.com/paper/2010.10754/) (2020)
- [Pack and Detect: Fast Object Detection in Videos Using Region-of-Interest Packing](https://mlsanity.com/paper/1809.01701/) (2018)
- [MOSAIC: Spatially-Multiplexed Edge AI Optimization over Multiple Concurrent Video Sensing Streams](https://mlsanity.com/paper/2305.03222/) (2023)

## Abstract (arXiv)

In this paper, we present Mondrian, an edge system that enables high-performance object detection on high-resolution video streams. Many lightweight models and system optimization techniques have been proposed for resource-constrained devices, but they do not fully utilize the potential of the accelerators over dynamic, high-resolution videos. To enable such capability, we devise a novel Compressive Packed Inference to minimize per-pixel processing costs by selectively determining the necessary pixels to process and combining them to maximize processing parallelism. In particular, our system quickly extracts ROIs and dynamically shrinks them, reflecting the effect of the fast-changing characteristics of objects and scenes. It then intelligently combines such scaled ROIs into large canvases to maximize the utilization of inference accelerators such as GPU. Evaluation across various datasets, models, and devices shows Mondrian outperforms state-of-the-art baselines (e.g., input rescaling, ROI extractions, ROI extractions+batching) by 15.0-19.7% higher accuracy, leading to ×6.65 higher throughput than frame-wise inference for processing various 1080p video streams. The code is available at https://github.com/snuhcs/mondrian.