mlsanityData to 14 Oct 2026

BitROM: Weight Reload-Free CiROM Architecture Towards Billion-Parameter 1.58-bit LLM Inference

arXiv 2509.08542 ↗First posted 10 Sep 2025cs.ARcited by 3

On the map: Efficient LLM inference and serving → Serving systems and request scheduling

Abstract (arXiv)

Compute-in-Read-Only-Memory (CiROM) accelerators offer outstanding energy efficiency for CNNs by eliminating runtime weight updates. However, their scalability to Large Language Models (LLMs) is fundamentally constrained by their vast parameter sizes. Notably, LLaMA-7B - the smallest model in LLaMA series - demands more than 1,000 cm2 of silicon area even in advanced CMOS nodes. This paper presents BitROM, the first CiROM-based accelerator that overcomes this limitation through co-design with BitNet's 1.58-bit quantization model, enabling practical and efficient LLM inference at the edge. BitROM introduces three key innovations: 1) a novel Bidirectional ROM Array that stores two ternary weights per transistor; 2) a Tri-Mode Local Accumulator optimized for ternary-weight computations; and 3) an integrated Decode-Refresh (DR) eDRAM that supports on-die KV-cache management, significantly reducing external memory access during decoding. In addition, BitROM integrates LoRA-based adapters to enable efficient transfer learning across various downstream tasks. Evaluated in 65nm CMOS, BitROM achieves 20.8 TOPS/W and a bit density of 4,967 kB/mm2 - offering a 10x improvement in area efficiency over prior digital CiROM designs. Moreover, the DR eDRAM contributes to a 43.6% reduction in external DRAM access, further enhancing deployment efficiency for LLMs in edge applications.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Serving systems and request scheduling, one of 12 directions in Efficient LLM inference and serving. Builds on 6 of its 22 references that are in the map. First posted on arXiv 10 Sep 2025.

Builds on

6 of 22 references.

References · 6

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  3. 2024The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bitscited by 308
  4. 2024BitNet a4.8: 4-bit Activations for 1-bit LLMscited by 16
  5. 2024ASiM: Modeling and Analyzing Inference Accuracy of SRAM-Based Analog CiM Circuitscited by 4
  6. 2022YOLoC: DeploY Large-Scale Neural Network by ROM-based Computing-in-Memory using ResiduaL Branch on a Chipcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026Physical Foundation Models: Fixed hardware implementations of large-scale neural networkscited by 1
  2. 2026Ankhdjet: An Open-Source Compiler for Mask-Programmed Ternary Compute-in-ROM on an Open PDKcited by 0
  3. 2026Implementation and Evaluation of BitNet Inference on a CGLA by Signed-Int4 Instructionscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Hardware Generation and Exploration of Lookup Table-Based Accelerators for 1.58-bit LLM Inference
  2. 2025TeLLMe: An Energy-Efficient Ternary LLM Accelerator for Prefilling and Decoding on Edge FPGAs
  3. 2025Binary Weight Multi-Bit Activation Quantization for Compute-in-Memory CNN Accelerators
  4. 2025ROMA: a Read-Only-Memory-based Accelerator for QLoRA-based On-Device LLM
  5. 2025Pushing up to the Limit of Memory Bandwidth and Capacity Utilization for Efficient LLM Decoding on Embedded FPGA
  6. 2025TeLLMe v2: An Efficient End-to-End Ternary LLM Prefill and Decode Accelerator with Table-Lookup Matmul on Edge FPGAs
  7. 2026VitaLLM: A Versatile, Ultra-Compact Ternary LLM Accelerator with Dependency-Aware Scheduling ↗
  8. 20241-bit AI Infra: Part 1.1, Fast and Lossless BitNet b1.58 Inference on CPUs