mlsanityData to 7 Oct 2026

MawForge: Memory-Bounded Expert Materialization for Local Mixture-of-Experts Inference

arXiv 2607.09686 ↗First posted 17 Jun 2026cs.LGcited by 0

On the map: Mixture-of-experts → Efficient MoE training and inference systems

also in 1 more
Abstract (arXiv)

Sparse Mixture-of-Experts (MoE) language models separate total parameter count from per-token active computation, but local inference systems often still require the full model, key-value cache, runtime buffers, and operatingsystem headroom to fit in fast memory. MawForge tests a different systems hypothesis: local MoE serving can be made practical on constrained unified-memory machines by storing the full model on disk, keeping common tensors resident, and materializing routed expert tensors into a bounded execution cache on demand. The central finding is that MawForge is effective as a bounded execution mechanism and measurement substrate for local MoE inference, but not as a cache-maximization policy. Performance depends on balancing expert reuse against resident footprint, KV-cache size, quantization, route locality, and macOS memory pressure.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Efficient MoE training and inference systems, one of 12 directions in Mixture-of-experts. Builds on 4 of its 6 references that are in the map. First posted on arXiv 17 Jun 2026.

Builds on

4 of 6 references.

References · 4

  1. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  2. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,014
  3. 2022Fast Inference from Transformers via Speculative Decodingcited by 1,615
  4. 2020GShard: Scaling Giant Models with Conditional Computation and Automatic Shardingcited by 1,601

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024ExpertFlow: Efficient Mixture-of-Experts Inference via Predictive Expert Caching and Token Scheduling
  2. 2025Accelerating Mixture-of-Expert Inference with Adaptive Expert Split Mechanism
  3. 2025Not All Models Suit Expert Offloading: On Local Routing Consistency of Mixture-of-Expert Models
  4. 2025Klotski: Efficient Mixture-of-Expert Inference via Expert-Aware Multi-Batch Pipeline
  5. 2023Pre-gated MoE: An Algorithm-System Co-Design for Fast and Scalable Mixture-of-Expert Inference
  6. 2025OD-MoE: On-Demand Expert Loading for Cacheless Edge-Distributed MoE Inference
  7. 2024HOBBIT: A Mixed Precision Expert Offloading System for Fast MoE Inference
  8. 2023Towards MoE Deployment: Mitigating Inefficiencies in Mixture-of-Expert (MoE) Inference