mlsanityData to 13 Oct 2026

BitNet b1.58 2B4T Technical Report

arXiv 2504.12285 ↗First posted 16 Apr 2025cs.CL cs.LGcited by 37

Not on the map. Why?

Abstract (arXiv)

We introduce BitNet b1.58 2B4T, the first open-source, native 1-bit Large Language Model (LLM) at the 2-billion parameter scale. Trained on a corpus of 4 trillion tokens, the model has been rigorously evaluated across benchmarks covering language understanding, mathematical reasoning, coding proficiency, and conversational ability. Our results demonstrate that BitNet b1.58 2B4T achieves performance on par with leading open-weight, full-precision LLMs of similar size, while offering significant advantages in computational efficiency, including substantially reduced memory footprint, energy consumption, and decoding latency. To facilitate further research and adoption, the model weights are released via Hugging Face along with open-source inference implementations for both GPU and CPU architectures.

Key facts. Cited by 37 papers in the map (data through 13 Oct 2026). Builds on 37 of its 43 references that are in the map. First posted on arXiv 16 Apr 2025.

Builds on

37 of 43 references.

References · 37

  1. 2017Attention Is All You Needcited by 60,106
  2. 2017Proximal Policy Optimization Algorithmscited by 14,592
  3. 2021Training Verifiers to Solve Math Word Problemscited by 9,916
  4. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,091
  5. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,492
  6. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,986
  7. 2020Measuring Massive Multitask Language Understandingcited by 7,876
  8. 2025Qwen2.5-VL Technical Reportcited by 5,499
  9. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,270
  10. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,628
  11. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,053
  12. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,949
25 more
  1. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,925
  2. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,843
  3. 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,163
  4. 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,133
  5. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,805
  6. 2020GLU Variants Improve Transformercited by 1,759
  7. 2025Gemma 3 Technical Reportcited by 1,748
  8. 2023Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generationcited by 1,362
  9. 2024The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scalecited by 1,126
  10. 2023WizardLM: Empowering large pre-trained language models to follow complex instructionscited by 1,120
  11. 2024MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategiescited by 708
  12. 2024WildChat: 1M ChatGPT Interaction Logs in the Wildcited by 616
  13. 2023LMSYS-Chat-1M: A Large-Scale Real-World LLM Conversation Datasetcited by 458
  14. 2023UltraFeedback: Boosting Language Models with Scaled AI Feedbackcited by 398
  15. 2025SmolLM2: When Smol Goes Big -- Data-Centric Training of a Small Language Modelcited by 321
  16. 2024The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bitscited by 305
  17. 2023BitNet: Scaling 1-bit Transformers for Large Language Modelscited by 186
  18. 2024MathScale: Scaling Instruction Tuning for Mathematical Reasoningcited by 144
  19. 2024Liger Kernel: Efficient Triton Kernels for LLM Trainingcited by 127
  20. 2024Synthetic Data (Almost) from Scratch: Generalized Instruction Tuning for Language Modelscited by 99
  21. 2019Quick and (not so) Dirty: Unsupervised Selection of Justification Sentences for Multi-hop Question Answeringcited by 59
  22. 2021PokeBNN: A Binary Pursuit of Lightweight Accuracycited by 49
  23. 2024Q-Sparse: All Large Language Models can be Fully Sparsely-Activatedcited by 25
  24. 2022Foundation Transformerscited by 19
  25. 2024BitNet a4.8: 4-bit Activations for 1-bit LLMscited by 16

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2025: 17; 2026: 20.

37 papers.

Sort:

By area:

  1. 2025BitVLA: 1-bit Vision-Language-Action Models for Robotics Manipulationcited by 43
  2. 2025Adaptive Prediction-Powered AutoEval with Reliability and Efficiency Guaranteescited by 10
  3. 2025PTQTP: Post-Training Quantization to Trit-Planes for Large Language Modelscited by 9
  4. 2025Tequila: Trapping-free Ternary Quantization for Large Language Modelscited by 9
  5. 2025Optimal Brain Restoration for Joint Quantization and Sparsification of LLMscited by 6
  6. 2025TENET: An Efficient Sparsity-Aware LUT-Centric Architecture for Ternary LLM Inference On Edgecited by 6
  7. 2026AngelSlim: A more accessible, comprehensive, and efficient toolkit for large model compressioncited by 6
  8. 2025iFairy: the First 2-bit Complex LLM with All Parameters in ±1, ± icited by 5
  9. 2026Sherry: Hardware-Efficient 1.25-Bit Ternary Quantization via Fine-grained Sparsificationcited by 5
  10. 2025DP-Adam-AC: Privacy-preserving Fine-Tuning of Localizable Language Models Using Adam Optimization with Adaptive Clippingcited by 3
  11. 2026Bit-by-Bit: Progressive QAT Strategy with Outlier Channel Splitting for Stable Low-Bit LLMscited by 3
  12. 2026Attend to Your Own Thoughts: Breaking the Barrier for Post-Training Quantization of Reasoning LLMs through the Lens of 1.58-Bit Quantizationcited by 3
25 more
  1. 2025BitNet Distillationcited by 2
  2. 2025Fairy2i: Training Complex LLMs from Real LLMs with All Parameters in ± 1, ± icited by 2
  3. 2025Vec-LUT: Vector Table Lookup for Parallel Ultra-Low-Bit LLM Inference on Edge Devicescited by 2
  4. 2026LC-QAT: Data-Efficient 2-Bit QAT for LLMs via Linear-Constrained Vector Quantizationcited by 2
  5. 2025COUNTDOWN: Contextually Sparse Activation Filtering Out Unnecessary Weights in Down Projectioncited by 1
  6. 2025Rethinking 1-bit Optimization Leveraging Pre-trained Large Language Modelscited by 1
  7. 2025BitSkip: An Empirical Analysis of Quantization and Early Exit Composition in Transformerscited by 1
  8. 2026TOM: A Ternary Read-only Memory Accelerator for LLM-powered Edge Intelligencecited by 1
  9. 2026VitaLLM: A Versatile and Tiny Accelerator for Mixed-Precision LLM Inference on Edge Devicescited by 1
  10. 2026Spike-Aware INT8 Execution for Spiking Language Models on Commodity CPUscited by 1
  11. 2026When Can Depth Replace Precision? A Resource Theory of Quantized Neural Computationcited by 1
  12. 2026Breaking the 1.58-bit Barrier for Ternary LLMscited by 1
  13. 2026TerMeZO: Ternary Sparse Zeroth-Order Optimization for Fine-tuning BitNet Models at the Edgecited by 1
  14. 2025Transforming Decoder-Only Transformers for Accurate WiFi-Telemetry Based Indoor Localizationcited by 0
  15. 2025Ultra-Quantisation: Efficient Embedding Search via 1.58-bit Encodingscited by 0
  16. 2025Rethinking Inference Placement for Deep Learning across Edge and Cloud Platforms: A Multi-Objective Optimization Perspective and Future Directionscited by 0
  17. 2026pQuant: Towards Effective Low-Bit Language Models via Decoupled Linear Quantization-Aware Trainingcited by 0
  18. 2026On the Expressive Power of Weight Quantization in Large Language Modelscited by 0
  19. 2026BitNet Text Embeddingscited by 0
  20. 2026Low-Rank Ternary Adaptation for Fine-Tuning Transformerscited by 0
  21. 2026Ankhdjet: An Open-Source Compiler for Mask-Programmed Ternary Compute-in-ROM on an Open PDKcited by 0
  22. 2026Activation Outliers Matter: Robust Recovery for Quantized Multimodal LLMscited by 0
  23. 2026Implementation and Evaluation of BitNet Inference on a CGLA by Signed-Int4 Instructionscited by 0
  24. 2026An Exploratory Ablation of a Small MLA--SSM Hybrid Language Modelcited by 0
  25. 2026Lost in the bf16 Cast: Exporting Ternary Language Models Can Revert Most Low-Learning-Rate Code Changescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 20241-bit AI Infra: Part 1.1, Fast and Lossless BitNet b1.58 Inference on CPUs
  2. 2025BitNet v2: Native 4-bit Activations with Hadamard Transformation for 1-bit LLMs
  3. 2024FBI-LLM: Scaling Up Fully Binarized LLMs from Scratch via Autoregressive Distillation
  4. 2024When are 1.58 bits enough? A Bottom-up Exploration of BitNet Quantization
  5. 2023Efficient LLM Inference on CPUs
  6. 2024DeepSeek LLM: Scaling Open-Source Language Models with Longtermism
  7. 2024GEB-1.3B: Open Lightweight Large Language Model
  8. 2024BitNet b1.58 Reloaded: State-of-the-art Performance Also on Smaller Networks