mlsanityData to 14 Oct 2026

Towards Narrowing the Generalization Gap in Deep Boolean Networks

arXiv 2409.05905 ↗First posted 6 Sep 2024cs.LG cs.CV cs.NEcited by 3

Not on the map. Why?

Abstract (arXiv)

The rapid growth of the size and complexity in deep neural networks has sharply increased computational demands, challenging their efficient deployment in real-world scenarios. Boolean networks, constructed with logic gates, offer a hardware-friendly alternative that could enable more efficient implementation. However, their ability to match the performance of traditional networks has remained uncertain. This paper explores strategies to enhance deep Boolean networks with the aim of surpassing their traditional counterparts. We propose novel methods, including logical skip connections and spatiality preserving sampling, and validate them on vision tasks using widely adopted datasets, demonstrating significant improvement over existing approaches. Our analysis shows how deep Boolean networks can maintain high performance while minimizing computational costs through 1-bit logic operations. These findings suggest that Boolean networks are a promising direction for efficient, high-performance deep learning models, with significant potential for advancing hardware-accelerated AI applications.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 22 of its 55 references that are in the map. First posted on arXiv 6 Sep 2024.

Builds on

22 of 55 references.

References · 22

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2016XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networkscited by 849
  4. 2019Green AIcited by 627
  5. 2015BinaryConnect: Training Deep Neural Networks with binary weights during propagationscited by 592
  6. 2017Scalable Training of Artificial Neural Networks with Adaptive Sparse Connectivity inspired by Network Sciencecited by 328
  7. 2021Thinking Like Transformerscited by 184
  8. 2016Why and When Can Deep -- but Not Shallow -- Networks Avoid the Curse of Dimensionality: a Reviewcited by 155
  9. 2016FINN: A Framework for Fast, Scalable Binarized Neural Network Inferencecited by 150
  10. 2020Training Binary Neural Networks with Real-to-Binary Convolutionscited by 112
  11. 2023Tracr: Compiled Transformers as a Laboratory for Interpretabilitycited by 104
  12. 2022FP8 Quantization: The Power of the Exponentcited by 88
10 more
  1. 2014On the Computational Efficiency of Training Neural Networkscited by 71
  2. 2022Deep Differentiable Logic Gate Networkscited by 67
  3. 2016Highway and Residual Networks learn Unrolled Iterative Estimationcited by 52
  4. 2022BiT: Robustly Binarized Multi-distilled Transformercited by 52
  5. 2023Scaling MLPs: A Tale of Inductive Biascited by 37
  6. 2023Towards Green AI in Fine-tuning Large Language Models via Adaptive Backpropagationcited by 12
  7. 2020Handling Missing Data in Decision Trees: A Probabilistic Approachcited by 9
  8. 2019Stochastic Feedforward Neural Networks: Universal Approximationcited by 7
  9. 2023Tailor: Altering Skip Connections for Resource-Efficient Inferencecited by 6
  10. 2022Robust Binary Models by Pruning Randomly-initialized Networkscited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 2.

3 papers.

  1. 2026Align Forward, Adapt Backward: Closing the Discretization Gap in Logic Gate Networkscited by 2
  2. 2025Differentiable Weightless Controllers: Learning Logic Circuits for Continuous Controlcited by 1
  3. 2026Fitting Multilinear Polynomials for Logic Gate Networkscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Learning with Boolean threshold functions
  2. 2025Practical Boolean Backpropagation
  3. 2026On the Depth Scalability of Logic Gate Networks
  4. 2021A comprehensive review of Binary Neural Network
  5. 2024An Open-source End-to-End Logic Optimization Framework for Large-scale Boolean Network with Reinforcement Learning
  6. 2026Polynomial Surrogate Training for Differentiable Ternary Logic Gate Networks
  7. 2025Characterising the Inductive Biases of Neural Networks on Boolean Data
  8. 2025Minimalist Softmax Attention Provably Learns Constrained Boolean Functions