mlsanityData to 7 Oct 2026

1-bit AI Infra: Part 1.1, Fast and Lossless BitNet b1.58 Inference on CPUs

arXiv 2410.16144 ↗First posted 21 Oct 2024cs.CLcited by 9

Not on the map. Why?

Abstract (arXiv)

Recent advances in 1-bit Large Language Models (LLMs), such as BitNet and BitNet b1.58, present a promising approach to enhancing the efficiency of LLMs in terms of speed and energy consumption. These developments also enable local LLM deployment across a broad range of devices. In this work, we introduce bitnet.cpp, a tailored software stack designed to unlock the full potential of 1-bit LLMs. Specifically, we develop a set of kernels to support fast and lossless inference of ternary BitNet b1.58 LLMs on CPUs. Extensive experiments demonstrate that bitnet.cpp achieves significant speedups, ranging from 2.37x to 6.17x on x86 CPUs and from 1.37x to 5.07x on ARM CPUs, across various model sizes. The code is available at https://github.com/microsoft/BitNet.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 4 of its 5 references that are in the map. First posted on arXiv 21 Oct 2024.

Builds on

4 of 5 references.

References · 4

  1. 2024The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bitscited by 289
  2. 2023BitNet: Scaling 1-bit Transformers for Large Language Modelscited by 175
  3. 2022LUT-GEMM: Quantized Matrix Multiplication based on LUTs for Efficient Inference in Large-Scale Generative Language Modelscited by 98
  4. 2024T-MAC: CPU Renaissance via Table Lookup for Low-Bit LLM Deployment on Edgecited by 47

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 7; 2026: 1.

9 papers.

  1. 2025Distilling LLM Agent into Small Models with Retrieval and Code Toolscited by 41
  2. 20241.58-bit FLUXcited by 31
  3. 2025Bitnet.cpp: Efficient Edge Inference for Ternary LLMscited by 17
  4. 2025Identifying Sensitive Weights via Post-quantization Integralcited by 10
  5. 2026Sherry: Hardware-Efficient 1.25-Bit Ternary Quantization via Fine-grained Sparsificationcited by 5
  6. 2025T-SAR: A Full-Stack Co-design for CPU-Only Ternary LLM Inference via In-Place SIMD ALU Reorganizationcited by 4
  7. 2025AWP: Activation-Aware Weight Pruning and Quantization with Projected Gradient Descentcited by 3
  8. 2025DP-Adam-AC: Privacy-preserving Fine-Tuning of Localizable Language Models Using Adam Optimization with Adaptive Clippingcited by 3
  9. 2025MoTE: Mixture of Ternary Experts for Memory-efficient Large Multimodal Modelscited by 1