GPT-OSS-20B: A Comprehensive Deployment-Centric Analysis of OpenAI's Open-Weight Mixture of Experts Model
On the map: Mixture-of-experts → Efficient MoE training and inference systems
Abstract (arXiv)
We present a single-GPU (H100, bf16) evaluation of GPT-OSS-20B (Mixture-of-Experts; 20.9B total, approx. 3.61B active) against dense baselines Qwen3-32B and Yi-34B across multiple dimensions. We measure true time-to-first-token (TTFT), full-decode throughput (TPOT), end-to-end latency percentiles, peak VRAM with past key values (PKV) held, and energy via a consistent nvidia-smi-based sampler. At a 2048-token context with 64-token decode, GPT-OSS-20B delivers higher decode throughput and tokens per Joule than dense baselines Qwen3-32B and Yi-34B, while substantially reducing peak VRAM and energy per 1000 generated tokens; its TTFT is higher due to MoE routing overhead. With only 17.3% of parameters active (3.61B of 20.9B), GPT-OSS-20B provides about 31.8% higher decode throughput and 25.8% lower energy per 1000 generated tokens than Qwen3-32B at 2048/64, while using 31.7% less peak VRAM. Normalized by active parameters, GPT-OSS-20B shows markedly stronger per-active-parameter efficiency (APE), underscoring MoE's deployment advantages. We do not evaluate accuracy; this is a deployment-focused study. We release code and consolidated results to enable replication and extension.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Efficient MoE training and inference systems, one of 12 directions in Mixture-of-experts. Builds on 37 of its 51 references that are in the map. First posted on arXiv 22 Aug 2025.
Builds on
37 of 51 references.
Closest ancestors · 1
Same direction, newest first
- 2022DeepSpeed-MoE: Advancing Mixture-of-Experts Inference and Training to Power Next-Generation AI Scale14 Jan
Other references · 36
- 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
- 2025Qwen3 Technical Reportcited by 8,820
- 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
- 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
- 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
- 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,509
- 2023Mistral 7Bcited by 3,164
- 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,073
- 2022Training Compute-Optimal Large Language Modelscited by 3,019
- 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,598
- 2024Qwen2 Technical Reportcited by 2,120
- 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,919
24 more
- 2024Mixtral of Expertscited by 1,823
- 2022Fast Inference from Transformers via Speculative Decodingcited by 1,653
- 2020GShard: Scaling Giant Models with Conditional Computation and Automatic Shardingcited by 1,631
- 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,386
- 2023A Survey on Evaluation of Large Language Modelscited by 1,242
- 2021Ethical and social risks of harm from Language Modelscited by 966
- 2022LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scalecited by 906
- 2024DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Modelscited by 867
- 2023Splitwise: Efficient generative LLM inference using phase splittingcited by 558
- 2024FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precisioncited by 519
- 2022ST-MoE: Designing Stable and Transferable Sparse Expert Modelscited by 399
- 2025FlashInfer: Efficient and Customizable Attention Engine for LLM Inference Servingcited by 258
- 2020Towards the Systematic Reporting of the Energy and Carbon Footprints of Machine Learningcited by 249
- 2024A Survey on Mixture of Experts in Large Language Modelscited by 237
- 2024QServe: W4A8KV4 Quantization and System Co-design for Efficient LLM Servingcited by 208
- 2019MLPerf Inference Benchmarkcited by 199
- 2022Eco2AI: carbon emissions tracking of machine learning models as the first step towards sustainable AIcited by 58
- 2025Energy Considerations of Large Language Model Inference and Efficiency Optimizationscited by 55
- 2024Mixture Compressor for Mixture-of-Experts LLMs Gains Morecited by 41
- 2024Speculative Decoding via Early-exiting for Faster LLM Inference with Thompson Sampling Control Mechanismcited by 32
- 2024Offline Energy-Optimal LLM Serving: Workload-Based Energy Models for LLM Inference on Heterogeneous Systemscited by 28
- 2025Characterizing LLM Inference Energy-Performance Tradeoffs across Workloads and GPU Scalingcited by 19
- 2025Unveiling Super Experts in Mixture-of-Experts Large Language Modelscited by 17
- 2023Part-time Power Measurements: nvidia-smi's Lack of Attentioncited by 15
Built on by
Citing papers per year: 2025: 1; 2026: 1.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2026WiSP: A Working-Set View of Mixture-of-Experts Serving on Extremely Low-Resource Hardware
- 2024Mixture of Experts with Mixture of Precisions for Tuning Quality of Service
- 2026Does Mixture-of-Experts Actually Help Inference on Consumer and Edge Hardware? An Empirical Study
- 2026Every Expert Counts: ExactMoE for Memory-Efficient W4A16 Inference
- 2024ExpertFlow: Efficient Mixture-of-Experts Inference via Predictive Expert Caching and Token Scheduling
- 2026FluxMoE: Decoupling Expert Residency for High-Performance MoE Serving
- 2024Optimizing Mixture-of-Experts Inference Time Combining Model Deployment and Communication Scheduling
- 2025MoETuner: Optimized Mixture of Expert Serving with Balanced Expert Placement and Token Routing