mlsanityData to 14 Oct 2026

PIM-LLM: A High-Throughput Hybrid PIM Architecture for 1-bit LLMs

arXiv 2504.01994 ↗First posted 31 Mar 2025cs.AR cs.AIcited by 1

Not on the map. Why?

Abstract (arXiv)

In this paper, we propose PIM-LLM, a hybrid architecture developed to accelerate 1-bit large language models (LLMs). PIM-LLM leverages analog processing-in-memory (PIM) architectures and digital systolic arrays to accelerate low-precision matrix multiplication (MatMul) operations in projection layers and high-precision MatMul operations in attention heads of 1-bit LLMs, respectively. Our design achieves up to roughly 80x improvement in tokens per second and a 70% increase in tokens per joule compared to conventional hardware accelerators. Additionally, PIM-LLM outperforms previous PIM-based LLM accelerators, setting a new benchmark with at least 2x and 5x improvement in GOPS and GOPS/W, respectively.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 22 of its 42 references that are in the map. First posted on arXiv 31 Mar 2025.

Builds on

22 of 42 references.

References · 22

  1. 2017Attention Is All You Needcited by 60,361
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  3. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
  4. 2016Gaussian Error Linear Units (GELUs)cited by 3,502
  5. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,386
  6. 2023OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Modelscited by 424
  7. 2024Contrastive Preference Optimization: Pushing the Boundaries of LLM Performance in Machine Translationcited by 394
  8. 2024The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bitscited by 308
  9. 2023BitNet: Scaling 1-bit Transformers for Large Language Modelscited by 189
  10. 2023Full Stack Optimization of Transformer Inference: a Surveycited by 80
  11. 2024OneBit: Towards Extremely Low-bit Large Language Modelscited by 79
  12. 2024Towards Greener LLMs: Bringing Energy-Efficiency to the Forefront of LLM Inferencecited by 61
10 more
  1. 2024Octopus v2: On-device language model for super agentcited by 47
  2. 2024ShiftAddLLM: Accelerating Pretrained LLMs via Post-Training Multiplication-Less Reparameterizationcited by 30
  3. 2023X-Former: In-Memory Acceleration of Transformerscited by 25
  4. 2024Memory Is All You Need: An Overview of Compute-in-Memory Architectures for Accelerating Large Language Model Inferencecited by 15
  5. 2024ConSmax: Hardware-Friendly Alternative Softmax with Learnable Parameterscited by 14
  6. 2021An In-Memory Analog Computing Co-Processor for Energy-Efficient CNN Inference on Mobile Devicescited by 9
  7. 2023PIM-GPT: A Hybrid Process-in-Memory Accelerator for Autoregressive Transformerscited by 8
  8. 2022MRAM-based Analog Sigmoid Function for In-memory Computingcited by 4
  9. 2023Heterogeneous Integration of In-Memory Analog Computing Architectures with Tensor Processing Unitscited by 4
  10. 2024Matmul or No Matmul in the Era of 1-bit LLMscited by 2

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025Sparse Attention Remapping with Clustering for Efficient LLM Decoding on PIMcited by 3

Similar, not cited

Close in wording, no citation either way

  1. 2025P3-LLM: An Integrated NPU-PIM Accelerator for Edge LLM Inference Using Hybrid Numerical Formats
  2. 2024New Solutions on LLM Acceleration, Optimization, and Application
  3. 2024PIMphony: Overcoming Bandwidth and Capacity Inefficiency in PIM-based Long-Context LLM Inference System
  4. 2025LLMPi: Optimizing LLMs for High-Throughput on Raspberry Pi
  5. 2024LlamaF: An Efficient Llama2 Architecture Accelerator on Embedded FPGAs
  6. 2025TeLLMe: An Energy-Efficient Ternary LLM Accelerator for Prefilling and Decoding on Edge FPGAs
  7. 2025FIGLUT: An Energy-Efficient Accelerator Design for FP-INT GEMM Using Look-Up Tables
  8. 2025LUT-LLM: Efficient Large Language Model Inference with Memory-based Computations on FPGAs