# SiliconBench: Speed, Memory, and Fidelity for LLM Serving on Unified-Memory Desktops

arXiv: https://arxiv.org/abs/2609.19169 · HTML: https://mlsanity.com/paper/2609.19169/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Serving systems and request scheduling, one of 12 directions in Efficient LLM inference and serving.
- Builds on 20 of its 33 references that are in the map.
- First posted on arXiv 12 Sep 2026.

## Placed in

- Efficient LLM inference and serving / [Serving systems and request scheduling](https://mlsanity.com/inference/serving-systems-and-request-scheduling/)

## Builds on (most cited first, 20 of 20)

- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code](https://mlsanity.com/paper/2403.07974/) (2024)
- [SGLang: Efficient Execution of Structured Language Model Programs](https://mlsanity.com/paper/2312.07104/) (2023)
- [Teaching Machines to Read and Comprehend](https://mlsanity.com/paper/1506.03340/) (2015)
- [Gated Delta Networks: Improving Mamba2 with Delta Rule](https://mlsanity.com/paper/2412.06464/) (2024)
- [Orca: Progressive Learning from Complex Explanation Traces of GPT-4](https://mlsanity.com/paper/2306.02707/) (2023)
- [You Only Cache Once: Decoder-Decoder Architectures for Language Models](https://mlsanity.com/paper/2405.05254/) (2024)
- [Petals: Collaborative Inference and Fine-tuning of Large Models](https://mlsanity.com/paper/2209.01188/) (2022)
- [SWE-bench Goes Live!](https://mlsanity.com/paper/2505.23419/) (2025)
- [ClawsBench: Evaluating Capability and Safety of LLM Productivity Agents in Simulated Workspaces](https://mlsanity.com/paper/2604.05172/) (2026)
- [Hermes 4 Technical Report](https://mlsanity.com/paper/2508.18255/) (2025)
- [LLM Output Drift: Cross-Provider Validation & Mitigation for Financial Workflows](https://mlsanity.com/paper/2511.07585/) (2025)
- [Production-Grade Local LLM Inference on Apple Silicon: A Comparative Study of MLX, MLC-LLM, Ollama, llama.cpp, and PyTorch MPS](https://mlsanity.com/paper/2511.05502/) (2025)
- [Native LLM and MLLM Inference at Scale on Apple Silicon](https://mlsanity.com/paper/2601.19139/) (2026)
- [Bench360: Benchmarking Local LLM Inference from 360 Degrees](https://mlsanity.com/paper/2511.16682/) (2025)
- [Prima.cpp: Fast 30-70B LLM Inference on Heterogeneous and Low-Resource Home Clusters](https://mlsanity.com/paper/2504.08791/) (2025)
- [Challenging GPU Dominance: When CPUs Outperform for On-Device LLM Inference](https://mlsanity.com/paper/2505.06461/) (2025)
- [Open-TQ-Metal: Fused Compressed-Domain Attention for Long-Context LLM Inference on Apple Silicon](https://mlsanity.com/paper/2604.16957/) (2026)
- [Towards Building Private LLMs: Exploring Multi-Node Expert Parallelism on Apple Silicon for Mixture-of-Experts Large Language Model](https://mlsanity.com/paper/2506.23635/) (2025)

## Abstract (arXiv)

Concurrent local LLM serving on unified-memory desktops must preserve memory headroom and output fidelity, which speed-only rankings overlook. We introduce SiliconBench, which evaluates nine Apple Silicon serving engines through three lenses: speed, memory, and fidelity. We evaluate chat and agent serving on Qwen3, Qwen3.5, and Gemma 4. We use a classification task to check for quality regressions against an NVIDIA reference. DGX Spark provides a complementary serving-performance reference. Three desiderata guide interpretation: serving architecture readiness, memory discipline, and multi-node scaling. On Qwen3-0.6B, vllm-metal alone more than doubles throughput on both workloads from concurrency 1 to 16. CUDA vLLM and SGLang show stronger concurrency scaling on the same prompts. Explicit memory budgets do not guarantee memory headroom: two stacks complete every request while memory use approaches physical capacity and throughput declines. The newer model architectures have narrower engine support. Their evaluated implementations match the fidelity reference. Only three stacks satisfy the completion, fidelity, and model-coverage gates. Comparisons on larger dense and MoE models reinforce the importance of scheduling prompt processing alongside ongoing generation: vllm-metal's packed prefill-decode path maintains lower first-token latency than omlx under concurrent load. In the tested two-machine configurations, tensor parallelism over Thunderbolt RDMA scales while pipeline parallelism over TCP regresses. We release benchmark code, per-run results, and maintenance journals, supported by a workflow combining bounded agent fixes with human review.