mlsanityData to 7 Oct 2026

NASiC: 3D NAND-based CAM-Selected Multibit CIM Architecture for Efficient On-Device Mixture-of-Experts LLM Inference

arXiv 2605.23294 ↗First posted 22 May 2026cs.ARcited by 0

On the map: Mixture-of-experts → Efficient MoE training and inference systems

Abstract (arXiv)

The Mixture-of-Experts (MoE) models have emerged as the state-of-the-art paradigm for scaling up large language models (LLMs) without proportionally increased computational cost. However, its on-device deployment faces a critical challenge due to the large memory requirement for storing all expert parameters. 3D NAND-based computing-in-memory (CIM) architectures uniquely offer high storage capacity and reduced data movement, while they are ill-suited for MoE models with dynamically sparse expert activation, leading to a degradation of effective computational parallelism, along with underutilization of multibit storage capability of Flash cells. In this work, we proposed a 3D NAND-based content addressable-selected CIM architecture, dubbed as NASiC, which is tailored to MoE models. By leveraging the intrinsic string structure of 3D NAND technology, NASiC fuses the dynamical expert selection through CAM-based masking mechanism and activated expert computation through CIM into a single computation cycle, eradicating redundant computation and enhancing computational parallelism. Moreover, circuit-level optimizations and multibit CIM cell are co-designed with proposed NASiC architecture, featuring block-wise parallel computation with in-situ signed multibit input and weight expansion, substantially improving the throughput and energy-efficiency of NAND CIM array, as well as the utilization of high-density 3D NAND technology for MoE models. With extensive experimental results, we demonstrate NASiC achieves 4-114.8x improved performance and 3.9-70x improved energy efficiency over state-of-the-art designs, along with high accuracy, showing its great potential for efficient on-device MoE LLM inference.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Efficient MoE training and inference systems, one of 12 directions in Mixture-of-experts. Builds on 20 of its 38 references that are in the map. First posted on arXiv 22 May 2026.

Builds on

20 of 38 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025SSD Offloading for LLM Mixture-of-Experts Weights Considered Harmful in Energy Efficiency9 Aug
  2. 2025Insights into DeepSeek-V3: Scaling Challenges and Reflections on Hardware for AI Architectures14 May
  3. 2023Pre-gated MoE: An Algorithm-System Co-Design for Fast and Scalable Mixture-of-Expert Inference23 Aug

Other references · 17

  1. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  2. 2020Scaling Laws for Neural Language Modelscited by 6,411
  3. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  4. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
  5. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,014
  6. 2020GShard: Scaling Giant Models with Conditional Computation and Automatic Shardingcited by 1,601
  7. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,338
  8. 2021GLaM: Efficient Scaling of Language Models with Mixture-of-Expertscited by 890
  9. 2024DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Modelscited by 834
  10. 2024A Survey on Mixture of Experts in Large Language Modelscited by 233
  11. 2024A Survey on Recent Advances in LLM-Based Multi-turn Dialogue Systemscited by 170
  12. 2025Perception, Reason, Think, and Plan: A Survey on Large Multimodal Reasoning Modelscited by 89
5 more
  1. 2023Towards MoE Deployment: Mitigating Inefficiencies in Mixture-of-Expert (MoE) Inferencecited by 36
  2. 2024A Survey: Collaborative Hardware and Software Design in the Era of Large Language Modelscited by 21
  3. 2024Cambricon-LLM: A Chiplet-Based Hybrid Architecture for On-Device Inference of 70B LLMcited by 17
  4. 2025Pangu Pro MoE: Mixture of Grouped Experts for Efficient Sparsitycited by 15
  5. 2025Collaborative Compression for Large-Scale MoE Deployment on Edgecited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026ThAME: 3D Memory-Enabled Heterogeneous Accelerator for LLM Mixture of Experts
  2. 2025A3D-MoE: Acceleration of Large Language Models with Mixture of Experts via 3D Heterogeneous Integration
  3. 2026NOVA: Technology-Architecture Co-Design of Near-Memory Processing for Attention-SSM-MoE Hybrid LLM Inference
  4. 2025P3-LLM: An Integrated NPU-PIM Accelerator for Edge LLM Inference Using Hybrid Numerical Formats
  5. 2026NVLLM: A 3D NAND-Centric Architecture Enabling Edge on-Device LLM Inference ↗
  6. 2026Beyond Capacity: Scalable MoE LLM Inference via High-Bandwidth Flash with Direct GPU and HBM Paths
  7. 2025KVNAND: Efficient On-Device Large Language Model Inference Using DRAM-Free In-Flash Computing
  8. 2026Mozart: Modularized and Efficient MoE Training on 3.5D Wafer-Scale Chiplet Architectures