mlsanityData to 7 Oct 2026

TileQ: Efficient Low-Rank Quantization of Mixture-of-Experts with 2D Tiling

arXiv 2605.09281 ↗First posted 10 May 2026cs.LGcited by 0

On the map: Mixture-of-experts → MoE large language models

Abstract (arXiv)

Mixture-of-Experts (MoE) models achieve remarkable performance by sparsely activating specialized experts, yet their massive parameters in experts pose significant challenges for deployment. While low-rank quantization offers a promising route to compress MoE models, existing methods still incur nonnegligible memory overhead and inference latency. To address these limitations, we propose textscTileQ, a fine-tuning-free post-training quantization (PTQ) method that employs 2D-tiling structured low-rank quantization to share low-rank factors across both input and output dimensions of MoE experts. Furthermore, we introduce an efficient inference technique for textscTileQ that fuses multiple low-rank expert computations into a single-pass operation, significantly improving hardware utilization. Experiments show that textscTileQ cuts down additional memory usage up to 10× and reduces inference latency to ~5% while preserving state-of-the-art accuracy.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in MoE large language models, one of 12 directions in Mixture-of-experts. Builds on 49 of its 60 references that are in the map. First posted on arXiv 10 May 2026.

Builds on

49 of 60 references.

Closest ancestors · 10

Same direction, newest first

  1. 2025MoQE: Improve Quantization Model performance via Mixture of Quantization Experts9 Aug
  2. 2025EAC-MoE: Expert-Selection Aware Compressor for Mixture-of-Experts Large Language Models3 Aug
  3. 2025MxMoE: Mixed-precision Quantization for MoE with Accuracy and Performance Co-Design9 May
  4. 2025MoEQuant: Enhancing Quantization for Mixture-of-Experts Large Language Models via Expert-Balanced Sampling and Affinity Guidance2 May
  5. 2024MoE-I2: Compressing Mixture of Experts Models through Inter-Expert Pruning and Intra-Expert Low-Rank Decomposition1 Nov
  6. 2024OLMoE: Open Mixture-of-Experts Language Models3 Sep
  7. 2024DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models11 Jan
  8. 2024Mixtral of Experts8 Jan
  9. 2023QMoE: Practical Sub-1-Bit Compression of Trillion-Parameter Models25 Oct
  10. 2023Mixture of Quantized Experts (MoQE): Complementary Effect of Low-bit Quantization and Robustness3 Oct

Other references · 39

  1. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  2. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  3. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  4. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
  5. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
  6. 2016Pointer Sentinel Mixture Modelscited by 2,992
  7. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,818
  8. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,838
  9. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,338
  10. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,234
  11. 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,107
  12. 2020Up or Down? Adaptive Rounding for Post-Training Quantizationcited by 600
27 more
  1. 2024QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMscited by 598
  2. 2024SpinQuant: LLM quantization with learned rotationscited by 446
  3. 2023OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Modelscited by 403
  4. 2022DeepSpeed-MoE: Advancing Mixture-of-Experts Inference and Training to Power Next-Generation AI Scalecited by 400
  5. 2023QuIP: 2-Bit Quantization of Large Language Models With Guaranteescited by 397
  6. 2023SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compressioncited by 359
  7. 2024QuIP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebookscited by 354
  8. 2024MoE-LLaVA: Mixture of Experts for Large Vision-Language Modelscited by 298
  9. 2024DuQuant: Distributing Outliers via Dual Transformation Makes Stronger Quantized LLMscited by 165
  10. 2020Adversarial Robustness: From Self-Supervised Pre-Training to Fine-Tuningcited by 152
  11. 2024EfficientQAT: Efficient Quantization-Aware Training for Large Language Modelscited by 123
  12. 2024QTIP: Quantization with Trellises and Incoherence Processingcited by 86
  13. 2024GPTVQ: The Blessing of Dimensionality for LLM Quantizationcited by 78
  14. 2024AffineQuant: Affine Transformation Quantization for Large Language Modelscited by 64
  15. 2018A Systematic Classification of Knowledge, Reasoning, and Context within the ARC Datasetcited by 54
  16. 2025GPTAQ: Efficient Finetuning-Free Quantization for Asymmetric Calibrationcited by 54
  17. 2024VPTQ: Extreme Low-bit Vector Post-Training Quantization for Large Language Modelscited by 53
  18. 2024LQER: Low-Rank Quantization Error Reconstruction for LLMscited by 39
  19. 2019Memory-Driven Mixed Low Precision Quantization For Enabling Deep Network Inference On Microcontrollerscited by 31
  20. 2024QERA: an Analytical Framework for Quantization Error Reconstructioncited by 21
  21. 2026Fixed Anchors Are Not Enough: Dynamic Retrieval and Persistent Homology for Dataset Distillationcited by 10
  22. 2025SpecQuant: Spectral Decomposition and Adaptive Truncation for Ultra-Low-Bit LLMs Quantizationcited by 9
  23. 2025MiLo: Efficient Quantized MoE Inference with Mixture of Low-Rank Compensatorscited by 7
  24. 2025Beyond Random: Automatic Inner-loop Optimization in Dataset Distillationcited by 4
  25. 2026BWLA: Breaking the Barrier of W1AX Post-Training Quantization for LLMscited by 4
  26. 2026LoPRo: Enhancing Low-Rank Quantization via Permuted Block-Wise Rotationcited by 3
  27. 2026FLRQ: Faster LLM Quantization with Flexible Low-Rank Matrix Sketchingcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024QuantMoE-Bench: Examining Post-Training Quantization for Mixture-of-Experts
  2. 2026Efficient Quantization of Mixture-of-Experts with Theoretical Generalization Guarantees
  3. 2025MC#: Mixture Compressor for Mixture-of-Experts Large Models
  4. 2026AlphaQ: Calibration-Free Bit Allocation for Mixture-of-Experts Quantization
  5. 2025Dynamic Expert Quantization for Scalable Mixture-of-Experts Inference
  6. 2026Q-Strata: Hierarchical Bit Allocation for Mixed-Precision Quantization of Mixture-of-Experts LLMs
  7. 2024Mixture Compressor for Mixture-of-Experts LLMs Gains More
  8. 2026Hardware-Native Joint Sparse-Quantization for Trillion-Scale Mixture-of-Experts