# Cascadia: A Control-Plane-Free Alternative to Hyperconverged AI Infrastructure

arXiv: https://arxiv.org/abs/2609.38697 · HTML: https://mlsanity.com/paper/2609.38697/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Serving systems and request scheduling, one of 12 directions in Efficient LLM inference and serving.
- Builds on 16 of its 63 references that are in the map.
- First posted on arXiv 30 Sep 2026.

## Placed in

- Efficient LLM inference and serving / [Serving systems and request scheduling](https://mlsanity.com/inference/serving-systems-and-request-scheduling/)

## Builds on (most cited first, 16 of 16)

- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving](https://mlsanity.com/paper/2401.09670/) (2024)
- [Splitwise: Efficient generative LLM inference using phase splitting](https://mlsanity.com/paper/2311.18677/) (2023)
- [Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Serve](https://mlsanity.com/paper/2403.02310/) (2024)
- [PowerInfer: Fast Large Language Model Serving with a Consumer-grade GPU](https://mlsanity.com/paper/2312.12456/) (2023)
- [Mooncake: A KVCache-centric Disaggregated Architecture for LLM Serving](https://mlsanity.com/paper/2407.00079/) (2024)
- [Petals: Collaborative Inference and Fine-tuning of Large Models](https://mlsanity.com/paper/2209.01188/) (2022)
- [Parallax: Efficient LLM Inference Service over Decentralized Environment](https://mlsanity.com/paper/2509.26182/) (2025)
- [TPI-LLM: Serving 70B-scale LLMs Efficiently on Low-resource Edge Devices](https://mlsanity.com/paper/2410.00531/) (2024)
- [Faster MoE LLM Inference for Extremely Large Models](https://mlsanity.com/paper/2505.03531/) (2025)
- [Model-Distributed Inference for Large Language Models at the Edge](https://mlsanity.com/paper/2505.18164/) (2025)
- [TD-Pipe: Temporally-Disaggregated Pipeline Parallelism Architecture for High-Throughput LLM Inference](https://mlsanity.com/paper/2506.10470/) (2025)
- [CHESS: Optimizing LLM Inference via Channel-Wise Thresholding and Selective Sparsification](https://mlsanity.com/paper/2409.01366/) (2024)
- [SmallThinker: A Family of Efficient Large Language Models Natively Trained for Local Deployment](https://mlsanity.com/paper/2507.20984/) (2025)
- [Prima.cpp: Fast 30-70B LLM Inference on Heterogeneous and Low-Resource Home Clusters](https://mlsanity.com/paper/2504.08791/) (2025)
- [Pre-Compiled Pipeline Shards for Distributed LLM Inference on Intel AI PC Fleets](https://mlsanity.com/paper/2608.19147/) (2026)

## Abstract (arXiv)

We present Cascadia, a system for serving large language models on fleets of commodity Intel AIPCs using their CPU, integrated-GPU, and NPU resources. Every node embeds ingress, scheduling, and execution; inference requests require no dedicated routing control plane. Nodes join a libp2p QUIC mesh using CA-issued ed25519 admission certificates, gossip signed capabilities, exchange live load over direct peer streams, and route OpenAI-compatible requests to eligible peers. An operator-run certificate authority handles admission and fleet management outside the inference path. Three serving modes share one interface: whole-model execution on one node, load-balanced replicas, and pipeline-sharded chains using the compilation and speculative decoding mechanism of our companion paper. Optional KV-cache mobility reuses compatible conversation prefixes after a routing move, with cold recomputation on a miss. Signed response receipts and hash-chained logs support provenance and audit. A three-node Phi-3.5-mini NPU testbed delivered 3.10x the response throughput of its one-node configuration under ten concurrent requests; a separate four-node deployment recorded 4.06x the throughput of direct single-node serving. Paired latency observations, runtime measurements, and internal functional checks characterize the tested configurations. We compare Cascadia with IBM, Nutanix, VMware, and HPE platforms on deployment footprint, hardware requirements, scheduling, scaling, licensing, and trust, using vendor documentation. The paper repository provides benchmark scripts, curated measurements, and a claim-to-evidence map.