# Neuromorphic Diffusion Language Models: Addressing Compute and Memory Bottlenecks via Sparsity and Block Denoising

arXiv: https://arxiv.org/abs/2607.24841 · HTML: https://mlsanity.com/paper/2607.24841/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Discrete and text diffusion, one of 14 directions in Diffusion models.
- Builds on 8 of its 14 references that are in the map.
- First posted on arXiv 24 Jul 2026.

## Placed in

- Diffusion models / [Discrete and text diffusion](https://mlsanity.com/diffusion/discrete-and-text-diffusion/)

## Builds on (most cited first, 8 of 8)

- [Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Models](https://mlsanity.com/paper/2503.09573/) (2025)
- [Spikformer: When Spiking Neural Network Meets Transformer](https://mlsanity.com/paper/2209.15425/) (2022)
- [Encoder-Decoder Diffusion Language Models for Efficient Training and Inference](https://mlsanity.com/paper/2510.22852/) (2025)
- [Optimising Event-Driven Spiking Neural Network with Regularisation and Cutoff](https://mlsanity.com/paper/2301.09522/) (2023)
- [How to keep pushing ML accelerator performance? Know your rooflines!](https://mlsanity.com/paper/2505.16346/) (2025)
- [SpikingBrain: Spiking Brain-inspired Large Models](https://mlsanity.com/paper/2509.05276/) (2025)
- [RooflineBench: A Benchmarking Framework for On-Device LLMs via Roofline Analysis](https://mlsanity.com/paper/2602.11506/) (2026)
- [Modern Neuromorphic AI: From Intra-Token to Inter-Token Processing](https://mlsanity.com/paper/2601.00245/) (2026)

## Abstract (arXiv)

Autoregressive (AR) large language models (LLMs) are inherently inefficient at inference time because each generated token requires accessing the full set of model parameters, leading to low operational intensity and high energy consumption. Masked diffusion language models (MDLMs) partially address this limitation for memory-bound settings by allowing multiple tokens to be generated per parameter access. In order to further enhance inference efficiency on modern platforms with extensive in-chip memory, this work proposes neuromorphic MDLMs (N-MDLMs), which integrate block diffusion with spike-based neuromorphic computation to jointly improve throughput and energy efficiency. While block diffusion increases token throughput by producing multiple tokens per parameter access, spike-induced sparsity reduces effective parameter traffic and computations by skipping inactive channels. To analyze the synergistic effect of sparsity and diffusion, we develop a token-level roofline-inspired model that captures the combined impact of block-parallel generation and spike sparsity on decoding efficiency. Experimental results on translation tasks show that, thanks to spike-induced sparsity, N-MDLMs achieve substantial improvements in energy efficiency and throughput even in compute-bound platforms for which MDLMs would fail to improve over AR-LLMs.