mlsanityData to 7 Oct 2026

Cascadia: A Control-Plane-Free Alternative to Hyperconverged AI Infrastructure

arXiv 2609.38697 ↗First posted 30 Sep 2026cs.DC cs.AIcited by 0

On the map: Efficient LLM inference and serving → Serving systems and request scheduling also fits: Efficient reasoning and agent inference

Abstract (arXiv)

We present Cascadia, a system for serving large language models on fleets of commodity Intel AIPCs using their CPU, integrated-GPU, and NPU resources. Every node embeds ingress, scheduling, and execution; inference requests require no dedicated routing control plane. Nodes join a libp2p QUIC mesh using CA-issued ed25519 admission certificates, gossip signed capabilities, exchange live load over direct peer streams, and route OpenAI-compatible requests to eligible peers. An operator-run certificate authority handles admission and fleet management outside the inference path. Three serving modes share one interface: whole-model execution on one node, load-balanced replicas, and pipeline-sharded chains using the compilation and speculative decoding mechanism of our companion paper. Optional KV-cache mobility reuses compatible conversation prefixes after a routing move, with cold recomputation on a miss. Signed response receipts and hash-chained logs support provenance and audit. A three-node Phi-3.5-mini NPU testbed delivered 3.10x the response throughput of its one-node configuration under ten concurrent requests; a separate four-node deployment recorded 4.06x the throughput of direct single-node serving. Paired latency observations, runtime measurements, and internal functional checks characterize the tested configurations. We compare Cascadia with IBM, Nutanix, VMware, and HPE platforms on deployment footprint, hardware requirements, scheduling, scaling, licensing, and trust, using vendor documentation. The paper repository provides benchmark scripts, curated measurements, and a claim-to-evidence map.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Serving systems and request scheduling, one of 12 directions in Efficient LLM inference and serving. Builds on 16 of its 63 references that are in the map. First posted on arXiv 30 Sep 2026.

Builds on

16 of 63 references.

Closest ancestors · 5

Same direction, newest first

  1. 2026Pre-Compiled Pipeline Shards for Distributed LLM Inference on Intel AI PC Fleets19 Aug
  2. 2025Parallax: Efficient LLM Inference Service over Decentralized Environment30 Sep
  3. 2024Mooncake: A KVCache-centric Disaggregated Architecture for LLM Serving24 Jun
  4. 2024Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Serve4 Mar
  5. 2024DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving18 Jan

Other references · 11

  1. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  2. 2023Splitwise: Efficient generative LLM inference using phase splittingcited by 536
  3. 2023PowerInfer: Fast Large Language Model Serving with a Consumer-grade GPUcited by 210
  4. 2022Petals: Collaborative Inference and Fine-tuning of Large Modelscited by 84
  5. 2024TPI-LLM: Serving 70B-scale LLMs Efficiently on Low-resource Edge Devicescited by 9
  6. 2025Faster MoE LLM Inference for Extremely Large Modelscited by 9
  7. 2025Model-Distributed Inference for Large Language Models at the Edgecited by 8
  8. 2025TD-Pipe: Temporally-Disaggregated Pipeline Parallelism Architecture for High-Throughput LLM Inferencecited by 8
  9. 2024CHESS: Optimizing LLM Inference via Channel-Wise Thresholding and Selective Sparsificationcited by 6
  10. 2025SmallThinker: A Family of Efficient Large Language Models Natively Trained for Local Deploymentcited by 4
  11. 2025Prima.cpp: Fast 30-70B LLM Inference on Heterogeneous and Low-Resource Home Clusterscited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Unified AI Gateway: A Framework for Joint Model Routing and KV Cache Management
  2. 2026SwiftCache: Efficient LLM Serving for Multi-turn Conversations with Heterogeneous KV Cache Sharing
  3. 2026Cascadia: Resident 975B MoE Inference on Eleven AI PCs
  4. 2025BanaServe: Unified KV Cache and Dynamic Module Migration for Balancing Disaggregated LLM Serving in AI Infrastructure
  5. 2026ICaRus: Identical Cache Reuse for Efficient Multi Model Inference
  6. 2025Cascadia: An Efficient Cascade Serving System for Large Language Models
  7. 2026XWind: A Cross-site Router for Large Language Model Inference Serving at Renewable Energy Farms
  8. 2026CALVO: Improve Serving Efficiency for LLM Inferences with Intense Network Demands