# User-Assisted Collaborative Distributed Inference for Efficient QoS-Aware Autoscaling

arXiv: https://arxiv.org/abs/2608.11840 · HTML: https://mlsanity.com/paper/2608.11840/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 3 of its 29 references that are in the map.
- First posted on arXiv 12 Aug 2026.

## Builds on (most cited first, 3 of 3)

- [Making AI Less "Thirsty": Uncovering and Addressing the Secret Water Footprint of AI Models](https://mlsanity.com/paper/2304.03271/) (2023)
- [Scalability Optimization in Cloud-Based AI Inference Services: Strategies for Real-Time Load Balancing and Automated Scaling](https://mlsanity.com/paper/2504.15296/) (2025)
- [Optimizing Multi-DNN Inference on Mobile Devices through Heterogeneous Processor Co-Execution](https://mlsanity.com/paper/2503.21109/) (2025)

## Abstract (arXiv)

Growing demand for artificial intelligence (AI) inference services requires scalable infrastructure, yet centralized serving costs rise with demand. We propose a collaborative distributed inference system combining dedicated infrastructure with resources contributed by service users. Dedicated resources provide baseline capacity for maintaining quality of service (QoS), while volunteered resources absorb increasing demand without proportional growth in centralized infrastructure. To capture stochastic and dynamic interactions among users, resources, tasks, and policies, we develop a high-dimensional generative Markov model with structured temporal factorization. The model supports simulation and provides a foundation for task scheduling and QoS-aware resource allocation optimization. We evaluate the system across user populations, resource capacities, and centralized and distributed scheduling policies. Simulations show that distributed scheduling becomes increasingly advantageous as the user population grows, improving request completion and P99 latency while substantially reducing dedicated resource consumption. These results demonstrate the feasibility of user-assisted collaborative inference for infrastructure-efficient autoscaling.