mlsanityData to 7 Oct 2026

Neuromorphic Principles for Efficient Large Language Models on Intel Loihi 2

arXiv 2503.18002 ↗First posted 12 Feb 2025cs.NE cs.AI cs.ARcited by 6

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) deliver impressive performance but require large amounts of energy. In this work, we present a MatMul-free LLM architecture adapted for Intel's neuromorphic processor, Loihi 2. Our approach leverages Loihi 2's support for low-precision, event-driven computation and stateful processing. Our hardware-aware quantized model on GPU demonstrates that a 370M parameter MatMul-free model can be quantized with no accuracy loss. Based on preliminary results, we report up to 3x higher throughput with 2x less energy, compared to transformer-based LLMs on an edge GPU, with significantly better scaling. Further hardware optimizations will increase throughput and decrease energy consumption. These results show the potential of neuromorphic hardware for efficient inference and pave the way for efficient reasoning models capable of generating complex, long-form text rapidly and cost-effectively.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 27 of its 36 references that are in the map. First posted on arXiv 12 Feb 2025.

Builds on

27 of 36 references.

References · 27

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  3. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,616
  4. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
  5. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,016
  6. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,818
  7. 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,114
  8. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,838
  9. 2019Root Mean Square Layer Normalizationcited by 1,425
  10. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,338
  11. 2016Language Modeling with Gated Convolutional Networkscited by 688
  12. 2021MetaFormer Is Actually What You Need for Visioncited by 523
15 more
  1. 2022Diagonal State Spaces are as Effective as Structured State Spacescited by 350
  2. 2024The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bitscited by 289
  3. 2024HGRN2: Gated Linear RNNs with State Expansioncited by 122
  4. 2023Hierarchically Gated Recurrent Neural Network for Sequence Modelingcited by 100
  5. 2021Efficient Neuromorphic Signal Processing with Loihi 2cited by 90
  6. 2023SpikeGPT: Generative Pre-trained Language Model with Spiking Neural Networkscited by 82
  7. 2021DNNFusion: Accelerating Deep Neural Networks Execution with Advanced Operator Fusioncited by 56
  8. 2023Efficient Video and Audio processing with Loihi 2cited by 28
  9. 2024ShiftAddLLM: Accelerating Pretrained LLMs via Post-Training Multiplication-Less Reparameterizationcited by 27
  10. 2024Quamba: A Post-Training Quantization Recipe for Selective State Space Modelscited by 25
  11. 2024Scalable MatMul-free Language Modelingcited by 21
  12. 2024Mamba-PTQ: Outlier Channels in Recurrent Large Language Modelscited by 15
  13. 2023Binarized Neural Machine Translationcited by 12
  14. 2024Q-S5: Towards Quantized State Space Modelscited by 9
  15. 2023Multiplication-Free Transformer Training via Piecewise Affine Operationscited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 4.

6 papers.

  1. 2025A Systematic Analysis of Hybrid Linear Attentioncited by 38
  2. 2025Compute-in-Memory Implementation of State Space Models for Event Sequence Processingcited by 6
  3. 2026Modern Neuromorphic AI: From Intra-Token to Inter-Token Processingcited by 2
  4. 2026SpikF-GO: Spiking Fourier Graph Operators for Multivariate Time Series Forecastingcited by 0
  5. 2026The Neural Division of Labor: Biologically-Inspired Modular Architectures for Robust Neuromorphic Computingcited by 0
  6. 2026Event-Driven Language Models with Sparse Neural Activity for Neuromorphic Hardwarecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026AI Hardware Accelerators for Large Language Models: Architectures and the Memory Wall ↗
  2. 2025LUT-LLM: Efficient Large Language Model Inference with Memory-based Computations on FPGAs
  3. 2026Understand and Accelerate Memory Processing Pipeline for Large Language Model Inference
  4. 2023Efficient LLM Inference on CPUs
  5. 2023Efficient LLM inference solution on Intel GPU
  6. 2024Inference Performance Optimization for Large Language Models on CPUs
  7. 2026The Economics of AI Decoding Chips: Rebalancing Compute, Capacity, and Bandwidth for Efficient LLM Inference
  8. 2025Make LLM Inference Affordable to Everyone: Augmenting GPU Memory with NDP-DIMM