# Resource Allocation for Multiuser Edge Inference with Batching and Early Exiting (Extended Version)

arXiv: https://arxiv.org/abs/2204.05223 · HTML: https://mlsanity.com/paper/2204.05223/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Apr 2022.

## Built on by (most cited first, 22 of 22)

- [Task-Oriented Sensing, Computation, and Communication Integration for Multi-Device Edge AI](https://mlsanity.com/paper/2207.00969/) (2022)
- [Green Edge AI: A Contemporary Survey](https://mlsanity.com/paper/2312.00333/) (2023)
- [Task-Oriented Over-the-Air Computation for Multi-Device Edge AI](https://mlsanity.com/paper/2211.01255/) (2022)
- [Edge Intelligence Optimization for Large Language Model Inference with Batching and Quantization](https://mlsanity.com/paper/2405.07140/) (2024)
- [The Larger the Merrier? Efficient Large AI Model Inference in Wireless Edge Networks](https://mlsanity.com/paper/2505.09214/) (2025)
- [Efficient Multiuser AI Downloading via Reusable Knowledge Broadcasting](https://mlsanity.com/paper/2307.15316/) (2023)
- [Collaborative Edge AI Inference over Cloud-RAN](https://mlsanity.com/paper/2404.06007/) (2024)
- [Revisiting Outage for Edge Inference Systems](https://mlsanity.com/paper/2504.03686/) (2025)
- [Optimal Expert Selection for Distributed Mixture-of-Experts at the Wireless Edge](https://mlsanity.com/paper/2503.13421/) (2025)
- [PartialLoading: User Scheduling and Bandwidth Allocation for Parameter-sharing Edge Inference](https://mlsanity.com/paper/2503.22982/) (2025)
- [Socialized Learning: A Survey of the Paradigm Shift for Edge Intelligence in Networked Systems](https://mlsanity.com/paper/2404.13348/) (2024)
- [Communication Efficient Cooperative Edge AI via Event-Triggered Computation Offloading](https://mlsanity.com/paper/2501.02001/) (2025)
- [Quantum Machine Learning for Secure Cooperative Multi-Layer Edge AI with Proportional Fairness](https://mlsanity.com/paper/2507.15145/) (2025)
- [Lightweight Remote Sensing Scene Classification on Edge Devices via Knowledge Distillation and Early-exit](https://mlsanity.com/paper/2507.20623/) (2025)
- [Optimizing video analytics inference pipelines: a case study](https://mlsanity.com/paper/2512.07009/) (2025)
- [Hierarchical Online-Scheduling for Energy-Efficient Split Inference with Progressive Transmission](https://mlsanity.com/paper/2601.08135/) (2026)
- [Channel-Adaptive Edge AI: Maximizing Inference Throughput by Adapting Computational Complexity to Channel States](https://mlsanity.com/paper/2603.03146/) (2026)
- [Joint Optimization of Training and Inference in Federated Edge Learning via Constrained Multi-Objective Deep Reinforcement Learning](https://mlsanity.com/paper/2605.25916/) (2026)
- [Multi-SPIN: Multi-Access Speculative Inference for Cooperative Token Generation at the Edge](https://mlsanity.com/paper/2606.04581/) (2026)
- [Goodput Maximization for Large Language Model Edge Inference: A Two-Phase Maskable PPO Approach](https://mlsanity.com/paper/2608.25543/) (2026)
- [MANE: A Multi-Path Adaptive Network for Edge Onloading of Deep Neural Networks](https://mlsanity.com/paper/2609.14660/) (2026)
- [Proportional-Fair Resource Allocation and Dual-Threshold Early-Exit Inference for Secure Cooperative Multi-Layer Edge Intelligence](https://mlsanity.com/paper/2609.15847/) (2026)

## Abstract (arXiv)

The deployment of inference services at the network edge, called edge inference, offloads computation-intensive inference tasks from mobile devices to edge servers, thereby enhancing the former's capabilities and battery lives. In a multiuser system, the joint allocation of communication-and-computation (C2) resources (i.e., scheduling and bandwidth allocation) is made challenging by adopting efficient inference techniques, batching and early exiting, and further complicated by the heterogeneity in users' requirements on accuracy and latency. Batching groups multiple tasks into one batch for parallel processing to reduce time-consuming memory access and thereby boosts the throughput (i.e., completed task per second). On the other hand, early exiting allows a task to exit from a deep-neural network without traversing the whole network to support a tradeoff between accuracy and latency. In this work, we study optimal C2 resource allocation with batching and early exiting, which is an NP-complete integer programming problem. A set of efficient algorithms are designed under the criterion of maximum throughput by tackling the challenge. Experimental results demonstrate that both optimal and sub-optimal C2 resource allocation algorithms can leverage integrated batching and early exiting to double the inference throughput compared with conventional schemes.