mlsanityData to 14 Oct 2026

The Serialized Bridge: Understanding and Recovering LLM Serving Performance under Blackwell GPU Confidential Computing

arXiv 2606.23969 ↗First posted 22 Jun 2026cs.DC cs.CR cs.PFcited by 0

On the map: Efficient LLM inference and serving → Serving systems and request scheduling

Abstract (arXiv)

GPU Confidential Computing (GPU-CC) now preserves GPU-local performance: on NVIDIA B300, BF16 matmul runs at 0.998x of non-confidential performance. Yet LLM serving under Intel TDX plus GPU-CC still loses 13-27% of throughput, and KV-cache restore latency can more than double. This paper studies that gap on two Blackwell platforms, RTX Pro 6000 and B300 HGX, and identifies its dominant cause: the confidential VM-GPU bridge, not GPU compute. We find that GPU-CC turns host/device movement into a serialized, high-setup-cost channel. Secure copies do not gain CUDA-stream concurrency within a context, asynchronous transfers block at the runtime boundary, and small crossings pay a fixed toll. This violates the assumptions of modern inference runtimes, where DMA is expected to be cheap, concurrent, and asynchronous. In vLLM dense decode, the gap closes around 44x-slower small alloc-and-copy operations; targeted patches reject alternative explanations. A scheduling flag recovers 57% of the gap, while a worker-thread drain recovers up to 92% in qualified high-concurrency runs. The same bridge model explains a +131% KV-restore penalty and a 34x model-load slowdown. Blackwell also changes the confidential tenancy unit. We qualify confidential multi-GPU NVSwitch tenants on B300, including 510 GB/s NVLink P2P inside a CVM and concurrent isolated tenants, and identify the remaining fabric-attestation gap for production confidential AI platforms.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Serving systems and request scheduling, one of 12 directions in Efficient LLM inference and serving. Builds on 5 of its 23 references that are in the map. First posted on arXiv 22 Jun 2026.

Builds on

5 of 23 references.

References · 5

  1. 2022Machine Learning with Confidential Computing: A Systematization of Knowledgecited by 15
  2. 2024PipeLLM: Fast and Confidential Large Language Model Services with Speculative Pipelined Encryptioncited by 10
  3. 2024Confidential Computing on NVIDIA Hopper GPUs: A Performance Benchmark Studycited by 8
  4. 2025Speeding up Model Loading with fastsafetensorscited by 3
  5. 2025Blueprint, Bootstrap, and Bridge: A Security Look at NVIDIA GPU Confidential Computingcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Private LLM Inference on Consumer Blackwell GPUs: A Practical Guide for Cost-Effective Local Deployment in SMEs
  2. 2025DuetServe: Harmonizing Prefill and Decode for LLM Serving via Adaptive GPU Multiplexing
  3. 2026FleetOpt: Analytical Fleet Provisioning for LLM Inference with Compress-and-Route as Implementation Mechanism
  4. 2024Saving GPU Hours in LLM Inference System Development and Online Workloads with Simulation and DBMS-Inspired Cache Replacement Policies
  5. 2025MultiPath Memory Access: Breaking Host-GPU Bandwidth Bottlenecks in LLM Services ↗
  6. 2026Fast Recovery for LLM Serving via Decoupled Device Memory Lifetime in Dynamo
  7. 2026LLMVisor: A Real-Time Latency Attribution Model for Multi-Tenant LLM Serving
  8. 2024ConServe: Fine-Grained GPU Harvesting for LLM Online and Offline Co-Serving