# DNN-Driven Compressive Offloading for Edge-Assisted Semantic Video Segmentation

arXiv: https://arxiv.org/abs/2203.14481 · HTML: https://mlsanity.com/paper/2203.14481/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 28 references that are in the map.
- First posted on arXiv 28 Mar 2022.

## Builds on (most cited first, 13 of 13)

- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018)
- [BiSeNet: Bilateral Segmentation Network for Real-time Semantic Segmentation](https://mlsanity.com/paper/1808.00897/) (2018)
- [Panoptic Feature Pyramid Networks](https://mlsanity.com/paper/1901.02446/) (2019)
- [Dilated Residual Networks](https://mlsanity.com/paper/1705.09914/) (2017)
- [Improving Semantic Segmentation via Video Propagation and Label Relaxation](https://mlsanity.com/paper/1812.01593/) (2018)
- [Fast Optical Flow using Dense Inverse Search](https://mlsanity.com/paper/1603.03590/) (2016)
- [Low-Latency Video Semantic Segmentation](https://mlsanity.com/paper/1804.00389/) (2018)
- [Quantization Guided JPEG Artifact Correction](https://mlsanity.com/paper/2004.09320/) (2020)
- [Clockwork Convnets for Video Semantic Segmentation](https://mlsanity.com/paper/1608.03609/) (2016)
- [Dynamic Video Segmentation Network](https://mlsanity.com/paper/1804.00931/) (2018)
- [Efficient Video Semantic Segmentation with Labels Propagation and Refinement](https://mlsanity.com/paper/1912.11844/) (2019)
- [Task-relevant Representation Learning for Networked Robotic Perception](https://mlsanity.com/paper/2011.03216/) (2020)

## Built on by (most cited first, 3 of 3)

- [Edge Video Analytics: A Survey on Applications, Systems and Enabling Techniques](https://mlsanity.com/paper/2211.15751/) (2022)
- [SALINA: Towards Sustainable Live Sonar Analytics in Wild Ecosystems](https://mlsanity.com/paper/2410.19742/) (2024)
- [OneAdapt: Fast Configuration Adaptation for Video Analytics Applications via Backpropagation](https://mlsanity.com/paper/2310.02422/) (2023)

## Abstract (arXiv)

Deep learning has shown impressive performance in semantic segmentation, but it is still unaffordable for resource-constrained mobile devices. While offloading computation tasks is promising, the high traffic demands overwhelm the limited bandwidth. Existing compression algorithms are not fit for semantic segmentation, as the lack of obvious and concentrated regions of interest (RoIs) forces the adoption of uniform compression strategies, leading to low compression ratios or accuracy. This paper introduces STAC, a DNN-driven compression scheme tailored for edge-assisted semantic video segmentation. STAC is the first to exploit DNN's gradients as spatial sensitivity metrics for spatial adaptive compression and achieves superior compression ratio and accuracy. Yet, it is challenging to adapt this content-customized compression to videos. Practical issues include varying spatial sensitivity and huge bandwidth consumption for compression strategy feedback and offloading. We tackle these issues through a spatiotemporal adaptive scheme, which (1) takes partial strategy generation operations offline to reduce communication load, and (2) propagates compression strategies and segmentation results across frames through dense optical flow, and adaptively offloads keyframes to accommodate video content. We implement STAC on a commodity mobile device. Experiments show that STAC can save up to 20.95% of bandwidth without losing accuracy, compared to the state-of-the-art algorithm.