# LEAP: LLM Inference on Scalable PIM-NoC Architecture with Balanced Dataflow and Fine-Grained Parallelism

arXiv: https://arxiv.org/abs/2509.14781 · HTML: https://mlsanity.com/paper/2509.14781/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Sep 2025.

## Built on by (most cited first, 4 of 4)

- [PICNIC: Silicon Photonic Interconnected Chiplets with Computational Network and In-memory Computing for LLM Inference Acceleration](https://mlsanity.com/paper/2511.04036/) (2025)
- [SHIELD: A Segmented Hierarchical Memory Architecture for Energy-Efficient LLM Inference on Edge NPUs](https://mlsanity.com/paper/2604.07396/) (2026)
- [CHIPSMORE: Compute-in-Interconnect and -Memory Chiplets for Multi-Mode Multi-Request LLM Inference Acceleration](https://mlsanity.com/paper/2608.30509/) (2026)
- [LLM Inference on IMC-NoC Architecture with Balanced Dataflow and Fine-Grained Parallelism](https://mlsanity.com/paper/2609.00857/) (2026)

## Abstract (arXiv)

Large language model (LLM) inference has been a prevalent demand in daily life and industries. The large tensor sizes and computing complexities in LLMs have brought challenges to memory, computing, and databus. This paper proposes a computation/memory/communication co-designed non-von Neumann accelerator by aggregating processing-in-memory (PIM) and computational network-on-chip (NoC), termed LEAP. The matrix multiplications in LLMs are assigned to PIM or NoC based on the data dynamicity to maximize data locality. Model partition and mapping are optimized by heuristic design space exploration. Dedicated fine-grained parallelism and tiling techniques enable high-throughput dataflow across the distributed resources in PIM and NoC. The architecture is evaluated on Llama 1B/8B/13B models and shows ~2.55× throughput (tokens/sec) improvement and ~71.94× energy efficiency (tokens/Joule) boost compared to the A100 GPU.