mlsanityData to 7 Oct 2026

FP8-Flow-MoE: A Casting-Free FP8 Recipe without Double Quantization Error

arXiv 2511.02302 ↗First posted 4 Nov 2025cs.LG cs.AIcited by 1

On the map: Mixture-of-experts → Efficient MoE training and inference systems

Abstract (arXiv)

Training large Mixture-of-Experts (MoE) models remains computationally prohibitive due to their extreme compute and memory demands. Although low-precision training promises to accelerate computation and reduce memory footprint, existing implementations still rely on BF16-dominated dataflows with frequent quantize-dequantize (Q/DQ) conversions. These redundant casts erode much of FP8's theoretical efficiency. However, naively removing these casts by keeping dataflows entirely in FP8 introduces double quantization error: tensors quantized along different dimensions accumulate inconsistent scaling factors, degrading numerical stability. We propose FP8-Flow-MoE, an FP8 training recipe featuring a quantization-consistent FP8-centric dataflow with a scaling-aware transpose and fused FP8 operators that streamline computation and eliminate explicit cast operations from 12 to 2. Evaluations on a 671B-parameter MoE model demonstrate up to 21% higher throughput and 16.5 GB lower memory usage per GPU compared to BF16 and na"ive FP8 baselines, while maintaining stable convergence. We provide a plug-and-play FP8 recipe compatible with TransformerEngine and Megatron-LM, which will be open-sourced soon.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Efficient MoE training and inference systems, one of 12 directions in Mixture-of-experts. Builds on 13 of its 20 references that are in the map. First posted on arXiv 4 Nov 2025.

Builds on

13 of 20 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025Insights into DeepSeek-V3: Scaling Challenges and Reflections on Hardware for AI Architectures14 May
  2. 2022DeepSpeed-MoE: Advancing Mixture-of-Experts Inference and Training to Power Next-Generation AI Scale14 Jan

Other references · 11

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  5. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,014
  6. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  7. 2020GShard: Scaling Giant Models with Conditional Computation and Automatic Shardingcited by 1,601
  8. 2017Mixed Precision Trainingcited by 1,135
  9. 2022FP8 Formats for Deep Learningcited by 187
  10. 2018Training Deep Neural Networks with 8-bit Floating Point Numberscited by 163
  11. 2023FP8-LM: Training FP8 Large Language Modelscited by 75

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Practical FP4 Training for Large-Scale MoE Models on Hopper GPUscited by 2

Similar, not cited

Close in wording, no citation either way

  1. 2026Scalable Training of Mixture-of-Experts Models with Megatron Core
  2. 2025Dynamic Expert Quantization for Scalable Mixture-of-Experts Inference
  3. 2025MiLo: Efficient Quantized MoE Inference with Mixture of Low-Rank Compensators
  4. 2026Hardware-Native Joint Sparse-Quantization for Trillion-Scale Mixture-of-Experts
  5. 2026MoEBlaze: Breaking the Memory Wall for Efficient MoE Training on Modern GPUs
  6. 2026Hyperparameter Scaling Laws Across MoE Sparsity
  7. 2025EAQuant: Enhancing Post-Training Quantization for MoE Models via Expert-Aware Optimization
  8. 2026Mixture-of-Parallelisms: Towards Memory-Efficient Training Stack for Mixture-of-Experts Models