# GEMEL: Model Merging for Memory-Efficient, Real-Time Video Analytics at the Edge

arXiv: https://arxiv.org/abs/2201.07705 · HTML: https://mlsanity.com/paper/2201.07705/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- Builds on 39 of its 112 references that are in the map.
- First posted on arXiv 19 Jan 2022.

## Builds on (most cited first, 30 of 39)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [DARTS: Differentiable Architecture Search](https://mlsanity.com/paper/1806.09055/) (2018)
- [How transferable are features in deep neural networks?](https://mlsanity.com/paper/1411.1792/) (2014)
- [Transformers in Vision: A Survey](https://mlsanity.com/paper/2101.01169/) (2021)
- [Efficient Neural Architecture Search via Parameter Sharing](https://mlsanity.com/paper/1802.03268/) (2018)
- [Learning and Generalization in Overparameterized Neural Networks, Going Beyond Two Layers](https://mlsanity.com/paper/1811.04918/) (2018)
- [Learning Overparameterized Neural Networks via Stochastic Gradient Descent on Structured Data](https://mlsanity.com/paper/1808.01204/) (2018)
- [ZeRO-Infinity: Breaking the GPU Memory Wall for Extreme Scale Deep Learning](https://mlsanity.com/paper/2104.07857/) (2021)
- [Clipper: A Low-Latency Online Prediction Serving System](https://mlsanity.com/paper/1612.03079/) (2016)
- [AdaShare: Learning What To Share For Efficient Deep Multi-Task Learning](https://mlsanity.com/paper/1911.12423/) (2019)
- [Ultimate SLAM? Combining Events, Images, and IMU for Robust Visual SLAM in HDR and High Speed Scenarios](https://mlsanity.com/paper/1709.06310/) (2017)
- [Serving DNNs like Clockwork: Performance Predictability from the Bottom Up](https://mlsanity.com/paper/2006.02464/) (2020)
- [TensorFlow-Serving: Flexible, High-Performance ML Serving](https://mlsanity.com/paper/1712.06139/) (2017)
- [Analysis of Large-Scale Multi-Tenant GPU Clusters for DNN Training Workloads](https://mlsanity.com/paper/1901.05758/) (2019)
- [Branched Multi-Task Networks: Deciding What Layers To Share](https://mlsanity.com/paper/1904.02920/) (2019)
- [Ekya: Continuous Learning of Video Analytics Models on Edge Compute Servers](https://mlsanity.com/paper/2012.10557/) (2020)
- [Focus: Querying Large Video Datasets with Low Latency and Low Cost](https://mlsanity.com/paper/1801.03493/) (2018)
- [Survey on Deep Neural Networks in Speech and Vision Systems](https://mlsanity.com/paper/1908.07656/) (2019)
- [Scaling Video Analytics on Constrained Edge Nodes](https://mlsanity.com/paper/1905.13536/) (2019)
- [ModelDiff: Testing-Based DNN Similarity Comparison for Model Reuse Detection](https://mlsanity.com/paper/2106.08890/) (2021)
- [Evolution of Image Segmentation using Deep Convolutional Neural Network: A Survey](https://mlsanity.com/paper/2001.04074/) (2020)
- [Incremental Learning in Deep Convolutional Neural Networks Using Partial Network Sharing](https://mlsanity.com/paper/1712.02719/) (2017)

## Built on by (most cited first, 30 of 32)

- [Pushing Large Language Models to the 6G Edge: Vision, Challenges, and Opportunities](https://mlsanity.com/paper/2309.16739/) (2023)
- [Apparate: Rethinking Early Exits to Tame Latency-Throughput Tensions in ML Serving](https://mlsanity.com/paper/2312.05385/) (2023)
- [SuperServe: Fine-Grained Inference Serving for Unpredictable Workloads](https://mlsanity.com/paper/2312.16733/) (2023)
- [TrimCaching: Parameter-sharing AI Model Caching in Wireless Edge Networks](https://mlsanity.com/paper/2405.03990/) (2024)
- [Turbo: Opportunistic Enhancement for Edge Video Analytics](https://mlsanity.com/paper/2207.00172/) (2022)
- [Edge Video Analytics: A Survey on Applications, Systems and Enabling Techniques](https://mlsanity.com/paper/2211.15751/) (2022)
- [MadEye: Boosting Live Video Analytics Accuracy with Adaptive Camera Configurations](https://mlsanity.com/paper/2304.02101/) (2023)
- [DC-CCL: Device-Cloud Collaborative Controlled Learning for Large Vision Models](https://mlsanity.com/paper/2303.10361/) (2023)
- [MOSEL: Inference Serving Using Dynamic Modality Selection](https://mlsanity.com/paper/2310.18481/) (2023)
- [AVA: Towards Agentic Video Analytics with Vision Language Models](https://mlsanity.com/paper/2505.00254/) (2025)
- [PartialLoading: User Scheduling and Bandwidth Allocation for Parameter-sharing Edge Inference](https://mlsanity.com/paper/2503.22982/) (2025)
- [Multi-Modal Data-Enhanced Foundation Models for Prediction and Control in Wireless Networks: A Survey](https://mlsanity.com/paper/2601.03181/) (2026)
- [Representation Similarity: A Better Guidance of DNN Layer Sharing for Edge Computing without Training](https://mlsanity.com/paper/2410.11233/) (2024)
- [Exploring Model Kinship for Merging Large Language Models](https://mlsanity.com/paper/2410.12613/) (2024)
- [E4: Energy-Efficient DNN Inference for Edge Video Analytics Via Early-Exit and DVFS](https://mlsanity.com/paper/2503.04865/) (2025)
- [Real-Time Privacy Preservation for Robot Visual Perception](https://mlsanity.com/paper/2505.05519/) (2025)
- [Recursive Offloading for LLM Serving in Multi-tier Networks](https://mlsanity.com/paper/2505.16502/) (2025)
- [ECLM: Efficient Edge-Cloud Collaborative Learning with Continuous Environment Adaptation](https://mlsanity.com/paper/2311.11083/) (2023)
- [BiSwift: Bandwidth Orchestrator for Multi-Stream Video Analytics on Edge](https://mlsanity.com/paper/2312.15740/) (2023)
- [Enabling Cross-Camera Collaboration for Video Analytics on Distributed Smart Cameras](https://mlsanity.com/paper/2401.14132/) (2024)
- [StatsMerging: Statistics-Guided Model Merging via Task-Specific Teacher Distillation](https://mlsanity.com/paper/2506.04567/) (2025)
- [MGit: A Model Versioning and Management System](https://mlsanity.com/paper/2307.07507/) (2023)
- [Automatic and Efficient Customization of Neural Networks for ML Applications](https://mlsanity.com/paper/2310.04685/) (2023)
- [UnifiedNN: Efficient Neural Network Training on the Cloud](https://mlsanity.com/paper/2408.01331/) (2024)
- [CloudEye: A New Paradigm of Video Analysis System for Mobile Visual Scenarios](https://mlsanity.com/paper/2410.18399/) (2024)
- [FusedInf: Efficient Swapping of DNN Models for On-Demand Serverless Inference Services on the Edge](https://mlsanity.com/paper/2410.21120/) (2024)
- [Responsive DNN Adaptation for Video Analytics against Environment Shift via Hierarchical Mobile-Cloud Collaborations](https://mlsanity.com/paper/2505.00745/) (2025)
- [XTransfer: Modality-Agnostic Few-Shot Model Transfer for Human Sensing at the Edge](https://mlsanity.com/paper/2506.22726/) (2025)
- [EPARA: Parallelizing Categorized AI Inference in Edge Clouds](https://mlsanity.com/paper/2511.00603/) (2025)
- [Optimizing video analytics inference pipelines: a case study](https://mlsanity.com/paper/2512.07009/) (2025)

## Abstract (arXiv)

Video analytics pipelines have steadily shifted to edge deployments to reduce bandwidth overheads and privacy violations, but in doing so, face an ever-growing resource tension. Most notably, edge-box GPUs lack the memory needed to concurrently house the growing number of (increasingly complex) models for real-time inference. Unfortunately, existing solutions that rely on time/space sharing of GPU resources are insufficient as the required swapping delays result in unacceptable frame drops and accuracy violations. We present model merging, a new memory management technique that exploits architectural similarities between edge vision models by judiciously sharing their layers (including weights) to reduce workload memory costs and swapping delays. Our system, GEMEL, efficiently integrates merging into existing pipelines by (1) leveraging several guiding observations about per-model memory usage and inter-layer dependencies to quickly identify fruitful and accuracy-preserving merging configurations, and (2) altering edge inference schedules to maximize merging benefits. Experiments across diverse workloads reveal that GEMEL reduces memory usage by up to 60.7%, and improves overall accuracy by 8-39% relative to time/space sharing alone.