# The Serialized Bridge: Understanding and Recovering LLM Serving Performance under Blackwell GPU Confidential Computing

arXiv: https://arxiv.org/abs/2606.23969 · HTML: https://mlsanity.com/paper/2606.23969/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Serving systems and request scheduling, one of 12 directions in Efficient LLM inference and serving.
- Builds on 5 of its 23 references that are in the map.
- First posted on arXiv 22 Jun 2026.

## Placed in

- Efficient LLM inference and serving / [Serving systems and request scheduling](https://mlsanity.com/inference/serving-systems-and-request-scheduling/)

## Builds on (most cited first, 5 of 5)

- [Machine Learning with Confidential Computing: A Systematization of Knowledge](https://mlsanity.com/paper/2208.10134/) (2022)
- [PipeLLM: Fast and Confidential Large Language Model Services with Speculative Pipelined Encryption](https://mlsanity.com/paper/2411.03357/) (2024)
- [Confidential Computing on NVIDIA Hopper GPUs: A Performance Benchmark Study](https://mlsanity.com/paper/2409.03992/) (2024)
- [Speeding up Model Loading with fastsafetensors](https://mlsanity.com/paper/2505.23072/) (2025)
- [Blueprint, Bootstrap, and Bridge: A Security Look at NVIDIA GPU Confidential Computing](https://mlsanity.com/paper/2507.02770/) (2025)

## Abstract (arXiv)

GPU Confidential Computing (GPU-CC) now preserves GPU-local performance: on NVIDIA B300, BF16 matmul runs at 0.998x of non-confidential performance. Yet LLM serving under Intel TDX plus GPU-CC still loses 13-27% of throughput, and KV-cache restore latency can more than double. This paper studies that gap on two Blackwell platforms, RTX Pro 6000 and B300 HGX, and identifies its dominant cause: the confidential VM-GPU bridge, not GPU compute. We find that GPU-CC turns host/device movement into a serialized, high-setup-cost channel. Secure copies do not gain CUDA-stream concurrency within a context, asynchronous transfers block at the runtime boundary, and small crossings pay a fixed toll. This violates the assumptions of modern inference runtimes, where DMA is expected to be cheap, concurrent, and asynchronous. In vLLM dense decode, the gap closes around 44x-slower small alloc-and-copy operations; targeted patches reject alternative explanations. A scheduling flag recovers 57% of the gap, while a worker-thread drain recovers up to 92% in qualified high-concurrency runs. The same bridge model explains a +131% KV-restore penalty and a 34x model-load slowdown. Blackwell also changes the confidential tenancy unit. We qualify confidential multi-GPU NVSwitch tenants on B300, including 510 GB/s NVLink P2P inside a CVM and concurrent isolated tenants, and identify the remaining fabric-attestation gap for production confidential AI platforms.