Highly Efficient and Effective LLMs with Multi-Boolean Architectures
Not on the map. Why?
Abstract (arXiv)
Weight binarization has emerged as a promising strategy to reduce the complexity of large language models (LLMs). Existing approaches fall into post-training binarization, which is simple but causes severe performance loss, and training-aware methods, which depend on full-precision latent weights, adding complexity and limiting efficiency. We propose a novel framework that represents LLMs with multi-kernel Boolean parameters and, for the first time, enables direct finetuning LMMs in the Boolean domain, eliminating the need for latent weights. This enhances representational capacity and dramatically reduces complexity during both finetuning and inference. Extensive experiments across diverse LLMs show our method outperforms recent ultra low-bit quantization and binarization techniques.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 56 of its 75 references that are in the map. First posted on arXiv 28 May 2025.
Builds on
56 of 75 references.
References · 56
- 2017Attention Is All You Needcited by 60,361
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
- 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
- 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
- 2015Distilling the Knowledge in a Neural Networkcited by 10,324
- 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
- 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,100
- 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
- 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
- 2023Qwen Technical Reportcited by 3,344
38 more
- 2016Pointer Sentinel Mixture Modelscited by 3,096
- 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,883
- 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,183
- 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,079
- 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,298
- 2023On-Policy Distillation of Language Models: Learning from Self-Generated Mistakescited by 939
- 2016XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networkscited by 849
- 2015BinaryConnect: Training Deep Neural Networks with binary weights during propagationscited by 592
- 2023FlexGen: High-Throughput Generative Inference of Large Language Models with a Single GPUcited by 575
- 2022ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformerscited by 526
- 2024GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projectioncited by 463
- 2023OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Modelscited by 424
- 2023QuIP: 2-Bit Quantization of Large Language Models With Guaranteescited by 414
- 2024QuIP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebookscited by 378
- 2023SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compressioncited by 367
- 2018Insights on representational similarity in neural networks with canonical correlationcited by 324
- 2023LLM-QAT: Data-Free Quantization Aware Training for Large Language Modelscited by 324
- 2024The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bitscited by 308
- 2023SqueezeLLM: Dense-and-Sparse Quantizationcited by 294
- 2022The case for 4-bit precision: k-bit Inference Scaling Lawscited by 251
- 2024The Unreasonable Ineffectiveness of the Deeper Layerscited by 226
- 2023BitNet: Scaling 1-bit Transformers for Large Language Modelscited by 189
- 2020BinaryBERT: Pushing the Limit of BERT Quantizationcited by 178
- 2024DistiLLM: Towards Streamlined Distillation for Large Language Modelscited by 163
- 2023Quantizable Transformers: Removing Outliers by Helping Attention Heads Do Nothingcited by 151
- 2024BiLLM: Pushing the Limit of Post-Training Quantization for LLMscited by 144
- 2023OWQ: Outlier-Aware Weight Quantization for Efficient Fine-Tuning and Inference of Large Language Modelscited by 114
- 2023f-Divergence Minimization for Sequence-Level Knowledge Distillationcited by 114
- 2024Scaling Laws for Precisioncited by 95
- 2024QTIP: Quantization with Trellises and Incoherence Processingcited by 91
- 2023PB-LLM: Partially Binarized Large Language Modelscited by 83
- 2024OneBit: Towards Extremely Low-bit Large Language Modelscited by 79
- 2022BiBERT: Accurate Fully Binarized BERTcited by 73
- 2022BiT: Robustly Binarized Multi-distilled Transformercited by 52
- 2024BitDelta: Your Fine-Tune May Only Be Worth One Bitcited by 39
- 2024DB-LLM: Accurate Dual-Binarization for Efficient LLMscited by 37
- 2024STBLLM: Breaking the 1-Bit Barrier with Structured Binary LLMscited by 31
- 2024ARB-LLM: Alternating Refined Binarizations for Large Language Modelscited by 31
50 most cited of 56 All 56 (JSON)
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2026All for 1-Bit: Towards Genuine 1-Bit Post-Training Quantization for LLMs
- 2025Achieving binary weight and activation for LLMs using Post-Training Quantization
- 2025Rethinking 1-bit Optimization Leveraging Pre-trained Large Language Models
- 2025SDQ-LLM: Sigma-Delta Quantization for 1-bit LLMs of any size
- 2026LBLLM: Lightweight Binarization of Large Language Models via Three-Stage Distillation
- 2024A Survey of Low-bit Large Language Models: Basics, Systems, and Algorithms
- 2023FineQuant: Unlocking Efficiency with Fine-Grained Weight-Only Quantization for LLMs
- 2024Any-Precision LLM: Low-Cost Deployment of Multiple, Different-Sized LLMs