mlsanityData to 14 Oct 2026

ThAME: 3D Memory-Enabled Heterogeneous Accelerator for LLM Mixture of Experts

arXiv 2607.17074 ↗First posted 19 Jul 2026cs.AR cs.AI cs.DCcited by 0

On the map: Mixture-of-experts → Efficient MoE training and inference systems

Abstract (arXiv)

Mixture of Experts (MoE) architectures have emerged as a dominant paradigm for scaling Large Language Models (LLMs). However, MoE inference on conventional hardware is constrained by three fundamental bottlenecks. These encompass the massive memory bandwidth required to fetch non-contiguous expert weights, the non-deterministic scatter-gather traffic generated by input-dependent token routing, and the tail-latency dependency imposed by synchronous expert output aggregation. To address these challenges, we propose ThAME, a three-dimensional (3D) heterogeneous multi-chiplet architecture for MoE inference. ThAME employs Ferroelectric Field-Effect Transistor (FeFET)-based non-volatile and DRAM-based volatile memory chiplets with a co-designed compute mapping strategy that aligns the distinct computational profiles of attention mechanisms and expert routing. Furthermore, we design a specialized Network-on-Chip communication backbone optimized to mitigate the bottlenecks associated with non-deterministic token routing traffic across the combinatorial space of input-dependent MoE traffic patterns. Experimental results demonstrate that ThAME outperforms state-of-the-art counterparts by up to 15.7x in terms of speedup and improves energy efficiency by up to 9.8x.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Efficient MoE training and inference systems, one of 12 directions in Mixture-of-experts. Builds on 10 of its 36 references that are in the map. First posted on arXiv 19 Jul 2026.

Builds on

10 of 36 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025A3D-MoE: Acceleration of Large Language Models with Mixture of Experts via 3D Heterogeneous Integration25 Jul
  2. 2025MixNet: A Runtime Reconfigurable Optical-Electrical Fabric for Distributed Mixture-of-Experts Training7 Jan
  3. 2022Accelerating Distributed MoE Training and Inference with Lina31 Oct

Other references · 7

  1. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  2. 2020DNN+NeuroSim V2.0: An End-to-End Benchmarking Framework for Compute-in-Memory Accelerators for On-chip Trainingcited by 33
  3. 2023X-Former: In-Memory Acceleration of Transformerscited by 25
  4. 2018Learning-based Application-Agnostic 3D NoC Design for Heterogeneous Manycore Systemscited by 8
  5. 2025CHIME: Chiplet-based Heterogeneous Near-Memory Acceleration for Edge Multimodal LLM Inferencecited by 3
  6. 2025Atleus: Accelerating Transformers on the Edge Enabled by 3D Heterogeneous Manycore Architecturescited by 2
  7. 2024HeTraX: Energy Efficient 3D Heterogeneous Manycore Architecture for Transformer Accelerationcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026NASiC: 3D NAND-based CAM-Selected Multibit CIM Architecture for Efficient On-Device Mixture-of-Experts LLM Inference
  2. 2026Mozart: Modularized and Efficient MoE Training on 3.5D Wafer-Scale Chiplet Architectures
  3. 2026HCRMap: Pressure-Aware Hot-Expert Residency Mapping for 3.5D MoE Chiplet Inference
  4. 2026Beyond Capacity: Scalable MoE LLM Inference via High-Bandwidth Flash with Direct GPU and HBM Paths
  5. 2026Expert Streaming: Accelerating Low-Batch MoE Inference via Multi-chiplet Architecture and Dynamic Expert Trajectory Scheduling
  6. 2025Accelerating Mixture-of-Expert Inference with Adaptive Expert Split Mechanism
  7. 2026EdgeXpert: An Edge Device for Memory-Efficient LLM Inference with Mixture-of-Experts and Speculative Decoding
  8. 2026CHIPSMORE: Compute-in-Interconnect and -Memory Chiplets for Multi-Mode Multi-Request LLM Inference Acceleration