# PartialLoading: User Scheduling and Bandwidth Allocation for Parameter-sharing Edge Inference

arXiv: https://arxiv.org/abs/2503.22982 · HTML: https://mlsanity.com/paper/2503.22982/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 62 references that are in the map.
- First posted on arXiv 29 Mar 2025.

## Builds on (most cited first, 30 of 35)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [How transferable are features in deep neural networks?](https://mlsanity.com/paper/1411.1792/) (2014)
- [A Comprehensive Survey on Transfer Learning](https://mlsanity.com/paper/1911.02685/) (2019)
- [SpotTune: Transfer Learning through Adaptive Fine-tuning](https://mlsanity.com/paper/1811.08737/) (2018)
- [On the Effectiveness of Adapter-based Tuning for Pretrained Language Model Adaptation](https://mlsanity.com/paper/2106.03164/) (2021)
- [On the Effectiveness of Parameter-Efficient Fine-Tuning](https://mlsanity.com/paper/2211.15583/) (2022)
- [Mobile Edge Intelligence for Large Language Models: A Contemporary Survey](https://mlsanity.com/paper/2407.18921/) (2024)
- [Communication-Efficient Edge AI: Algorithms and Systems](https://mlsanity.com/paper/2002.09668/) (2020)
- [Pushing Large Language Models to the 6G Edge: Vision, Challenges, and Opportunities](https://mlsanity.com/paper/2309.16739/) (2023)
- [Split Learning in 6G Edge Networks](https://mlsanity.com/paper/2306.12194/) (2023)
- [Communication-Computation Trade-Off in Resource-Constrained Edge Inference](https://mlsanity.com/paper/2006.02166/) (2020)
- [Optimal Resource Allocation for U-Shaped Parallel Split Learning](https://mlsanity.com/paper/2308.08896/) (2023)
- [GEMEL: Model Merging for Memory-Efficient, Real-Time Video Analytics at the Edge](https://mlsanity.com/paper/2201.07705/) (2022)
- [Collaborative Perception for Connected and Autonomous Driving: Challenges, Possible Solutions and Opportunities](https://mlsanity.com/paper/2401.01544/) (2024)
- [Mobile Edge Intelligence and Computing for the Internet of Vehicles](https://mlsanity.com/paper/1906.00400/) (2019)
- [Resource Allocation for Multiuser Edge Inference with Batching and Early Exiting (Extended Version)](https://mlsanity.com/paper/2204.05223/) (2022)
- [Realizing the Metaverse with Edge Intelligence: A Match Made in Heaven](https://mlsanity.com/paper/2201.01634/) (2022)
- [PACP: Priority-Aware Collaborative Perception for Connected and Autonomous Vehicles](https://mlsanity.com/paper/2404.06891/) (2024)
- [Prioritized Information Bottleneck Theoretic Framework with Distributed Online Learning for Edge Video Analytics](https://mlsanity.com/paper/2409.00146/) (2024)
- [Vehicle as a Service (VaaS): Leverage Vehicles to Build Service Networks and Capabilities for Smart Cities](https://mlsanity.com/paper/2304.11397/) (2023)
- [iGniter: Interference-Aware GPU Resource Provisioning for Predictable DNN Inference in the Cloud](https://mlsanity.com/paper/2211.01713/) (2022)
- [TrimCaching: Parameter-sharing AI Model Caching in Wireless Edge Networks](https://mlsanity.com/paper/2405.03990/) (2024)
- [Multi-user Co-inference with Batch Processing Capable Edge Server](https://mlsanity.com/paper/2206.06304/) (2022)
- [TrimCaching: Parameter-sharing Edge Caching for AI Model Downloading](https://mlsanity.com/paper/2404.14204/) (2024)
- [A Survey on Resource Management in Joint Communication and Computing-Embedded SAGIN](https://mlsanity.com/paper/2403.17400/) (2024)
- [Space-ground Fluid AI for 6G Edge Intelligence](https://mlsanity.com/paper/2411.15845/) (2024)
- [Edge Intelligence Optimization for Large Language Model Inference with Batching and Quantization](https://mlsanity.com/paper/2405.07140/) (2024)
- [Semantic Sleuth: Identifying Ponzi Contracts via Large Language Models](https://mlsanity.com/paper/2411.07498/) (2024)
- [DeepRT: A Soft Real Time Scheduler for Computer Vision Applications on the Edge](https://mlsanity.com/paper/2105.01803/) (2021)

## Built on by (most cited first, 4 of 4)

- [Visual Fidelity Index for Generative Semantic Communications with Critical Information Embedding](https://mlsanity.com/paper/2505.10405/) (2025)
- [Channel-Adaptive Edge AI: Maximizing Inference Throughput by Adapting Computational Complexity to Channel States](https://mlsanity.com/paper/2603.03146/) (2026)
- [Multi-SPIN: Multi-Access Speculative Inference for Cooperative Token Generation at the Edge](https://mlsanity.com/paper/2606.04581/) (2026)
- [BALANCE: Hybrid Autoregressive-Speculative LLM Inference at the Network Edge](https://mlsanity.com/paper/2608.05926/) (2026)

## Abstract (arXiv)

By provisioning inference offloading services, edge inference drives the rapid growth of AI applications at network edge. However, how to reduce the inference latency remains a significant challenge. To address this issue, we develop a parameter-sharing AI model loading (PartialLoading) framework for multi-user edge inference, which exploits two key insights: 1) the majority of latency arises from loading AI models into server GPU memory, and 2) different AI models can share a significant number of parameters, for which redundant loading should be avoided. Towards this end, we formulate a joint multi-user scheduling and spectrum bandwidth allocation problem to maximize task throughput by exploiting shared parameter blocks across models. The intuition is to judiciously schedule user requests to reuse the shared parameter blocks between consecutively loaded models, thereby reducing model loading time substantially. To facilitate solution finding, we decouple the problem into two sub-problems, i.e., user scheduling and bandwidth allocation, showing that solving them sequentially leads to the solution to the original problem. Due to the NP-hardness of the problem, we first study an important special case called the "backbone-sharing" case, and design a dynamic programming-based algorithm to obtain the optimal solution in polynomial time. For the general case, we propose a greedy heuristic to obtain the sub-optimal solution efficiently. Simulation results demonstrate that the proposed framework significantly improves task throughput under deadline constraints compared with user scheduling without exploiting parameter sharing.