mlsanityData to 14 Oct 2026

Implementation and Evaluation of BitNet Inference on a CGLA by Signed-Int4 Instructions

arXiv 2609.27453 ↗First posted 23 Sep 2026cs.ARcited by 0

Not on the map. Why?

Abstract (arXiv)

Large language model (LLM) inference transfers model weights and activations for every generated token, making memory traffic and its energy cost part of the decode path. BitNet b1.58 represents its low-bit weights by ternary values and uses integer activations. However, this arithmetic does not match conventional int8 or floating-point general matrix multiplication, and existing BitNet accelerators implement it in specialized datapaths. We instead map this operation to a CPU-Grounded Linear Array (CGLA), a programmable ASIC with explicit direct memory access, local memories, and reusable compiler-visible integer lanes. The mapping adds OP_SMA4 as a reusable signed-int4 multiply-accumulate instruction rather than a BitNet-only datapath. Each ternary weight occupies one signed 4-bit lane. Each int8 activation is split into two signed-int4 fragments and reconstructed by shift-and-add. Frequency scaling of the 145 MHz FPGA measurement to an 840 MHz 28 nm CGLA achieved 0.390 ns per signed-int4 product. We showed that CGLA-offloaded BitNet C++ execution measures 2.52 tokens/s.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 13 of its 16 references that are in the map. First posted on arXiv 23 Sep 2026.

Builds on

13 of 16 references.

References · 13

  1. 2024The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bitscited by 308
  2. 2023BitNet: Scaling 1-bit Transformers for Large Language Modelscited by 189
  3. 2017Bit Fusion: Bit-Level Dynamically Composable Architecture for Accelerating Deep Neural Networkscited by 109
  4. 2024T-MAC: CPU Renaissance via Table Lookup for Low-Bit LLM Deployment on Edgecited by 55
  5. 2025BitNet b1.58 2B4T Technical Reportcited by 38
  6. 2025Bitnet.cpp: Efficient Edge Inference for Ternary LLMscited by 22
  7. 2025TeLLMe: An Energy-Efficient Ternary LLM Accelerator for Prefilling and Decoding on Edge FPGAscited by 10
  8. 2025TerEffic: Highly Efficient Ternary LLM Inference on FPGAcited by 6
  9. 2025BitROM: Weight Reload-Free CiROM Architecture Towards Billion-Parameter 1.58-bit LLM Inferencecited by 3
  10. 2025NX-CGRA: A Programmable Hardware Accelerator for Core Transformer Algorithms on Edge Devicescited by 1
  11. 2025Efficient Kernel Mapping and Comprehensive System Evaluation of LLM Acceleration on a CGLAcited by 1
  12. 2026Hardware Generation and Exploration of Lookup Table-Based Accelerators for 1.58-bit LLM Inferencecited by 1
1 more
  1. 2026VitaLLM: A Versatile and Tiny Accelerator for Mixed-Precision LLM Inference on Edge Devicescited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 20241-bit AI Infra: Part 1.1, Fast and Lossless BitNet b1.58 Inference on CPUs
  2. 2026XtraMAC: An Efficient MAC Architecture for Mixed-Precision LLM Inference on FPGA
  3. 2025FPGA Co-Design for Efficient N:M Sparse and Quantized Model Inference
  4. 2025Hybrid Systolic Array Accelerator with Optimized Dataflow for Edge Large Language Model Inference
  5. 2026Implementation and Evaluation of NTT Arithmetic for ML-KEM on a CGLA ↗
  6. 2025TeLLMe v2: An Efficient End-to-End Ternary LLM Prefill and Decode Accelerator with Table-Lookup Matmul on Edge FPGAs
  7. 2024Anda: Unlocking Efficient LLM Inference with a Variable-Length Grouped Activation Data Format
  8. 2025LUT-LLM: Efficient Large Language Model Inference with Memory-based Computations on FPGAs