mlsanityData to 7 Oct 2026

SGLang: Efficient Execution of Structured Language Model Programs

arXiv 2312.07104 ↗First posted 12 Dec 2023cs.AI cs.PLcited by 1,107

Not on the map. Why?

Cited by 390 in Efficient LLM inference and serving, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Large language models (LLMs) are increasingly used for complex tasks that require multiple generation calls, advanced prompting techniques, control flow, and structured inputs/outputs. However, efficient systems are lacking for programming and executing these applications. We introduce SGLang, a system for efficient execution of complex language model programs. SGLang consists of a frontend language and a runtime. The frontend simplifies programming with primitives for generation and parallelism control. The runtime accelerates execution with novel optimizations like RadixAttention for KV cache reuse and compressed finite state machines for faster structured output decoding. Experiments show that SGLang achieves up to 6.4x higher throughput compared to state-of-the-art inference systems on various large language and multi-modal models on tasks including agent control, logical reasoning, few-shot learning benchmarks, JSON decoding, retrieval-augmented generation pipelines, and multi-turn chat. The code is publicly available at https://github.com/sgl-project/sglang

Key facts. Cited by 1,107 papers in the map (data through 7 Oct 2026). Builds on 36 of its 60 references that are in the map. First posted on arXiv 12 Dec 2023.

Builds on

36 of 60 references.

References · 36

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  5. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  6. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  7. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  8. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  9. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  10. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  11. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  12. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
24 more
  1. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
  2. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
  3. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
  4. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  5. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  6. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
  7. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,829
  8. 2022Competition-Level Code Generation with AlphaCodecited by 1,427
  9. 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,302
  10. 2023DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelinescited by 803
  11. 2023FlexGen: High-Throughput Generative Inference of Large Language Models with a Single GPUcited by 567
  12. 2022Efficiently Scaling Transformer Inferencecited by 529
  13. 2023Model Tells You What to Discard: Adaptive KV Cache Compression for LLMscited by 466
  14. 2023Scissorhands: Exploiting the Persistence of Importance Hypothesis for LLM KV Cache Compression at Test Timecited by 428
  15. 2023Cognitive Architectures for Language Agentscited by 422
  16. 2022DeepSpeed Inference: Enabling Efficient Inference of Transformer Models at Unprecedented Scalecited by 403
  17. 2023Prompt Cache: Modular Attention Reuse for Low-Latency Inferencecited by 255
  18. 2023Efficient Guided Generation for Large Language Modelscited by 192
  19. 2023An LLM Compiler for Parallel Function Callingcited by 170
  20. 2022Prompting Is Programming: A Query Language for Large Language Modelscited by 110
  21. 2023Branch-Solve-Merge Improves Large Language Model Evaluation and Generationcited by 104
  22. 2023Skeleton-of-Thought: Prompting LLMs for Efficient Parallel Generationcited by 103
  23. 2022Validating Large Language Models with ReLMcited by 33
  24. 2023Prompting Frameworks for Large Language Models: A Surveycited by 29

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 53; 2025: 314; 2026: 739.

1,107 papers.

Sort:

By area:

  1. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMscited by 1,193
  2. 2025From System 1 to System 2: A Survey of Reasoning Large Language Modelscited by 319
  3. 2025EAGLE-3: Scaling up Inference Acceleration of Large Language Models via Training-Time Testcited by 318
  4. 2024Training Software Engineering Agents and Verifiers with SWE-Gymcited by 300
  5. 2024Optimizing Instructions and Demonstrations for Multi-Stage Language Model Programscited by 270
  6. 2025AReaL: A Large-Scale Asynchronous Reinforcement Learning System for Language Reasoningcited by 234
  7. 2024Ada-KV: Optimizing KV Cache Eviction by Adaptive Budget Allocation for Efficient LLM Inferencecited by 230
  8. 2025MEM1: Learning to Synergize Memory and Reasoning for Efficient Long-Horizon Agentscited by 226
  9. 2024A Survey on Efficient Inference for Large Language Modelscited by 197
  10. 2024Mooncake: A KVCache-centric Disaggregated Architecture for LLM Servingcited by 193
  11. 2024VinePPO: Refining Credit Assignment in RL Training of LLMscited by 168
  12. 2026MiMo-V2-Flash Technical Reportcited by 163
38 more
  1. 2025Llama-Nemotron: Efficient Reasoning Modelscited by 151
  2. 2024A Survey on Large Language Model Acceleration based on KV Cache Managementcited by 125
  3. 2025olmOCR: Unlocking Trillions of Tokens in PDFs with Vision Language Modelscited by 120
  4. 2026DFlash: Block Diffusion for Flash Speculative Decodingcited by 120
  5. 2025LMCache: An Efficient KV Cache Layer for Enterprise-Scale LLM Inferencecited by 119
  6. 2025Soft Thinking: Unlocking the Reasoning Potential of LLMs in Continuous Concept Spacecited by 109
  7. 2025OpenCodeReasoning: Advancing Data Distillation for Competitive Codingcited by 106
  8. 2025Online Difficulty Filtering for Reasoning Oriented Reinforcement Learningcited by 100
  9. 2024vAttention: Dynamic Memory Management for Serving LLMs without PagedAttentioncited by 93
  10. 2024A Survey of Resource-efficient LLM and Multimodal Foundation Modelscited by 84
  11. 2024LLM and Simulation as Bilevel Optimizers: A New Paradigm to Advance Physical Scientific Discoverycited by 82
  12. 2025Process Reward Models for LLM Agents: Practical Framework and Directionscited by 81
  13. 2025MonkeyOCR: Document Parsing with a Structure-Recognition-Relation Triplet Paradigmcited by 81
  14. 2023Fairness in Serving Large Language Modelscited by 77
  15. 2025Cartridges: Lightweight and general-purpose long context representations via self-studycited by 74
  16. 2024MoE-Lightning: High-Throughput MoE Inference on Memory-constrained GPUscited by 73
  17. 2024XGrammar: Flexible and Efficient Structured Generation Engine for Large Language Modelscited by 73
  18. 2025Organize the Web: Constructing Domains Enhances Pre-Training Data Curationcited by 73
  19. 2025Revisiting the Test-Time Scaling of o1-like Models: Do they Truly Possess Test-Time Scaling Capabilities?cited by 69
  20. 2025Stabilizing MoE Reinforcement Learning by Aligning Training and Inference Routerscited by 67
  21. 2026Unifying Group-Relative and Self-Distillation Policy Optimization via Sample Routingcited by 65
  22. 2025Memory as Action: Autonomous Context Curation for Long-Horizon Agentic Taskscited by 62
  23. 2024SCBench: A KV Cache-Centric Analysis of Long-Context Methodscited by 59
  24. 2025AgentRL: Scaling Agentic Reinforcement Learning with a Multi-Turn, Multi-Task Frameworkcited by 58
  25. 2025LongVT: Incentivizing "Thinking with Long Videos" via Native Tool Callingcited by 58
  26. 2024Efficiently Scaling LLM Reasoning with Certaindexcited by 56
  27. 2025StreamRL: Scalable, Heterogeneous, and Elastic RL for LLMs with Disaggregated Stream Generationcited by 56
  28. 2025RoboMonkey: Scaling Test-Time Sampling and Verification for Vision-Language-Action Modelscited by 56
  29. 2025KVFlow: Efficient Prefix Caching for Accelerating LLM-Based Multi-Agent Workflowscited by 56
  30. 2025Breaking the Exploration Bottleneck: Rubric-Scaffolded Reinforcement Learning for General LLM Reasoningcited by 56
  31. 2025Baichuan-M2: Scaling Medical Capability with Large Verifier Systemcited by 55
  32. 2025rStar2-Agent: Agentic Reasoning Technical Reportcited by 54
  33. 2026PaddleOCR-VL-1.5: Towards a Multi-Task 0.9B VLM for Robust In-the-Wild Document Parsingcited by 54
  34. 2026Natural-Language Agent Harnessescited by 54
  35. 2025InftyThink: Breaking the Length Limits of Long-Context Reasoning in Large Language Modelscited by 53
  36. 2025ThetaEvolve: Test-time Learning on Open Problemscited by 52
  37. 2025Every Step Evolves: Scaling Reinforcement Learning for Trillion-Scale Thinking Modelcited by 50
  38. 2026CRISP: Compressed Reasoning via Iterative Self-Policy Distillationcited by 49

50 most cited of 1,107 All 1,107 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Minions: Accelerating Large Language Model Inference with Aggregated Speculative Execution
  2. 2026SEIS: Self-Evolving Inference Systems
  3. 2025Slm-mux: Orchestrating small language models for reasoning
  4. 2024GUIDE: A Global Unified Inference Engine for Deploying Large Language Models in Heterogeneous Environments
  5. 2025A*-Decoding: Token-Efficient Inference Scaling
  6. 2025Accelerating Large Language Model Inference via Early-Exiting Algorithms
  7. 2025Recursive Language Models
  8. 2025Small Language Models for Agentic Systems: A Survey of Architectures, Capabilities, and Deployment Trade offs