mlsanityData to 14 Oct 2026

SiliconBench: Speed, Memory, and Fidelity for LLM Serving on Unified-Memory Desktops

arXiv 2609.19169 ↗First posted 12 Sep 2026cs.AR cs.DC cs.SYcited by 0

On the map: Efficient LLM inference and serving → Serving systems and request scheduling

Abstract (arXiv)

Concurrent local LLM serving on unified-memory desktops must preserve memory headroom and output fidelity, which speed-only rankings overlook. We introduce SiliconBench, which evaluates nine Apple Silicon serving engines through three lenses: speed, memory, and fidelity. We evaluate chat and agent serving on Qwen3, Qwen3.5, and Gemma 4. We use a classification task to check for quality regressions against an NVIDIA reference. DGX Spark provides a complementary serving-performance reference. Three desiderata guide interpretation: serving architecture readiness, memory discipline, and multi-node scaling. On Qwen3-0.6B, vllm-metal alone more than doubles throughput on both workloads from concurrency 1 to 16. CUDA vLLM and SGLang show stronger concurrency scaling on the same prompts. Explicit memory budgets do not guarantee memory headroom: two stacks complete every request while memory use approaches physical capacity and throughput declines. The newer model architectures have narrower engine support. Their evaluated implementations match the fidelity reference. Only three stacks satisfy the completion, fidelity, and model-coverage gates. Comparisons on larger dense and MoE models reinforce the importance of scheduling prompt processing alongside ongoing generation: vllm-metal's packed prefill-decode path maintains lower first-token latency than omlx under concurrent load. In the tested two-machine configurations, tensor parallelism over Thunderbolt RDMA scales while pipeline parallelism over TCP regresses. We release benchmark code, per-run results, and maintenance journals, supported by a workflow combining bounded agent fixes with human review.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Serving systems and request scheduling, one of 12 directions in Efficient LLM inference and serving. Builds on 20 of its 33 references that are in the map. First posted on arXiv 12 Sep 2026.

Builds on

20 of 33 references.

References · 20

  1. 2025Qwen3 Technical Reportcited by 8,820
  2. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  3. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 2,004
  4. 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,141
  5. 2015Teaching Machines to Read and Comprehendcited by 1,049
  6. 2024Gated Delta Networks: Improving Mamba2 with Delta Rulecited by 532
  7. 2023Orca: Progressive Learning from Complex Explanation Traces of GPT-4cited by 379
  8. 2024You Only Cache Once: Decoder-Decoder Architectures for Language Modelscited by 144
  9. 2022Petals: Collaborative Inference and Fine-tuning of Large Modelscited by 87
  10. 2025SWE-bench Goes Live!cited by 80
  11. 2026ClawsBench: Evaluating Capability and Safety of LLM Productivity Agents in Simulated Workspacescited by 33
  12. 2025Hermes 4 Technical Reportcited by 8
8 more
  1. 2025LLM Output Drift: Cross-Provider Validation & Mitigation for Financial Workflowscited by 7
  2. 2025Production-Grade Local LLM Inference on Apple Silicon: A Comparative Study of MLX, MLC-LLM, Ollama, llama.cpp, and PyTorch MPScited by 5
  3. 2026Native LLM and MLLM Inference at Scale on Apple Siliconcited by 5
  4. 2025Bench360: Benchmarking Local LLM Inference from 360 Degreescited by 4
  5. 2025Prima.cpp: Fast 30-70B LLM Inference on Heterogeneous and Low-Resource Home Clusterscited by 3
  6. 2025Challenging GPU Dominance: When CPUs Outperform for On-Device LLM Inferencecited by 3
  7. 2026Open-TQ-Metal: Fused Compressed-Domain Attention for Long-Context LLM Inference on Apple Siliconcited by 2
  8. 2025Towards Building Private LLMs: Exploring Multi-Node Expert Parallelism on Apple Silicon for Mixture-of-Experts Large Language Modelcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025LLMServingSim2.0: A Unified Simulator for Heterogeneous Hardware and Serving Techniques in LLM Infrastructure
  2. 2026LLMServingSim 2.0: A Unified Simulator for Heterogeneous and Disaggregated LLM Serving Infrastructure
  3. 2026Hardware-Software Co-design for 3D-DRAM-based LLM Serving Accelerator
  4. 2025Systematic Exploration of Multi-core Architectures for Efficient LLM Serving using WaferAI-SIM
  5. 2026From Tensor Buffer to Distributed Memory Hierarchy: A Survey of KV Cache Management for LLM Serving
  6. 2025Sandwich: Joint Configuration Search and Hot-Switching for Efficient CPU LLM Serving
  7. 2026SLIM: Saturation-Aware Lightweight Performance Modeling for LLM Serving
  8. 2026Hydra: Phase-Aware Workload Characterization of LLM Inference across Edge SoC Generations, Backends, and Quantization Levels