mlsanityData to 14 Oct 2026

Tiny-QMoE

arXiv 2509.22951 ↗First posted 26 Sep 2025cs.PF cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

The QMoE model provides a practical approach for compression of massive Mixture-of-Experts (MoE) models. QMoE offers a solution geared towards memory limitations that often reach terabyte scales, and it has the advantage of working with high sparsity models which implicitly lend themselves to compression techniques. QMoE also has the advantage of only taking MoE models into account and does not evaluate its use with non mixture of expert systems. Although this prior attempt focuses on the limitations of large servers with the latest NVIDIA hardware which in the case of the H100 and V100 which have 80 GB of HBM (High Bandwidth Memory), what is not being considered is a significantly more constrained environment, such as in the case of mobile devices which may have in the case of the iPhone anywhere from 4 to 8 GB of unified memory which also needs to be shared with the operating system and additional processes. Although edge devices such as phones and laptops are becoming increasingly more computationally powerful, they are still not close to the level of advanced server machines such as NVIDIA. An additional constraint that we must consider is that of latency. The communication time of sending a request to an LLM server and then getting it back is an additional waiting time that can be removed. We may also want to use LLM technology in environments where there is no reliable network connection.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 5 of its 6 references that are in the map. First posted on arXiv 26 Sep 2025.

Builds on

5 of 6 references.

References · 5

  1. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  2. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  3. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  4. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,919
  5. 2023QMoE: Practical Sub-1-Bit Compression of Trillion-Parameter Modelscited by 42

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026LatentMoE: Toward Optimal Accuracy per FLOP and Parameter in Mixture of Experts
  2. 2025BuddyMoE: Exploiting Expert Redundancy to Accelerate Memory-Constrained Mixture-of-Experts Inference
  3. 2025MoBE: Mixture-of-Basis-Experts for Compressing MoE-based LLMs
  4. 2025Taming Latency-Memory Trade-Off in MoE-Based LLM Serving via Fine-Grained Expert Offloading
  5. 2026Benchmarking Composable Compression Techniques in Mixture-of-Experts LLMs
  6. 2025Collaborative Compression for Large-Scale MoE Deployment on Edge
  7. 2026MobileMoE: Scaling On-Device Mixture of Experts
  8. 2024On the Compressibility of Quantized Large Language Models