mlsanityData to 7 Oct 2026

Continuous Discovery of Vulnerabilities in LLM Serving Systems with Fuzzing

arXiv 2605.11202 ↗First posted 11 May 2026cs.CR cs.AI cs.LGcited by 1

On the map: Efficient LLM inference and serving → Serving systems and request scheduling

Abstract (arXiv)

LLM inference and serving systems have become security-critical infrastructure; however, many of their most concerning failures arise from the serving layer rather than from model behavior alone. Modern inference engines combine KV cache, batching, prefix sharing, speculative decoding, adapters, and multi-tenant scheduling, creating shared-state behavior that only emerges under realistic concurrent workloads and is missed by standard model, safety, and API tests. We present GRIEF, a greybox fuzzer for LLM inference engines that treats timed multi-request traces as first-class inputs, uses lightweight oracles to detect crashes, hangs, performance pathologies, and silent output corruption, and applies controlled replay with log-probability checks to confirm reproducible serving-layer failures. Across early campaigns on vLLM and SGLang, GRIEF discovers 15 vulnerabilities, 10 confirmed by engine developers, including 2 CVEs, spanning KV-cache isolation failures, cross-request performance interference, and crash or liveness bugs. These results show that concurrency, caching, and state reuse can induce silent cross-request contamination, noisy-neighbor denial of service, and delayed crashes without malformed inputs or explicit server errors, making concurrent serving behavior a first-class security and reliability boundary for LLM infrastructure.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Serving systems and request scheduling, one of 12 directions in Efficient LLM inference and serving. Builds on 24 of its 35 references that are in the map. First posted on arXiv 11 May 2026.

Builds on

24 of 35 references.

References · 24

  1. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  2. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  3. 2020Extracting Training Data from Large Language Modelscited by 2,144
  4. 2022Fast Inference from Transformers via Speculative Decodingcited by 1,615
  5. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,326
  6. 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,107
  7. 2024Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Headscited by 821
  8. 2023Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generationcited by 795
  9. 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Modelscited by 738
  10. 2022PAL: Program-aided Language Modelscited by 635
  11. 2024EAGLE: Speculative Sampling Requires Rethinking Feature Uncertaintycited by 564
  12. 2024JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Modelscited by 494
12 more
  1. 2022DeepSpeed-MoE: Advancing Mixture-of-Experts Inference and Training to Power Next-Generation AI Scalecited by 400
  2. 2025EAGLE-3: Scaling up Inference Acceleration of Large Language Models via Training-Time Testcited by 318
  3. 2023Fine-tuning Language Models for Factualitycited by 219
  4. 2019MLPerf Inference Benchmarkcited by 195
  5. 2024Stealing Part of a Production Language Modelcited by 152
  6. 2020Sponge Examples: Energy-Latency Attacks on Neural Networkscited by 108
  7. 2022Large Language Models are Zero-Shot Fuzzers: Fuzzing Deep-Learning Libraries via Large Language Modelscited by 102
  8. 2023Fuzz4All: Universal Fuzzing with Large Language Modelscited by 88
  9. 2024LLM-Inference-Bench: Inference Benchmarking of Large Language Models on AI Acceleratorscited by 31
  10. 2020Graph-Based Fuzz Testing for Deep Learning Inference Enginecited by 7
  11. 2025A First Look at Bugs in LLM Inference Enginescited by 5
  12. 2018Size-aware Sharding For Improving Tail Latencies in In-memory Key-value Storescited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Aborted but Not Forgotten: KV-Cache Retention Breaks Rollback Consistency in Language Agentscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026GhostServe: A Lightweight Checkpointing System in the Shadow for Fault-Tolerant LLM Serving
  2. 2026SeedSmith: LLM-Driven Seed Synthesis for Directed Fuzzing ↗
  3. 2026Antaeus: Hunting Repository-Level Logic Vulnerabilities via Context-Grounded LLM Reasoning ↗
  4. 2026FailureAtlas: A Taxonomy of Failure Modes in Multi-Provider LLM Serving Infrastructure
  5. 2026Concordia: JIT-Compiled Persistent-Kernel Checkpointing for Fault-Tolerant LLM Inference
  6. 2026Governing the KV Cache: Preventing Timing Side-Channel Leakage in Multi-Tenant LLM Inference
  7. 2026ChainFuzzer: Greybox Fuzzing for Workflow-Level Multi-Tool Vulnerabilities in LLM Agents
  8. 2026Rethinking Latency Denial-of-Service: Attacking the LLM Serving Framework, Not the Model