# Light Differentiable Logic Gate Networks

arXiv: https://arxiv.org/abs/2510.03250 · HTML: https://mlsanity.com/paper/2510.03250/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 26 references that are in the map.
- First posted on arXiv 26 Sep 2025.

## Builds on (most cited first, 11 of 11)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Deep Learning in Neural Networks: An Overview](https://mlsanity.com/paper/1404.7828/) (2014)
- [Learning and Generalization in Overparameterized Neural Networks, Going Beyond Two Layers](https://mlsanity.com/paper/1811.04918/) (2018)
- [Deep Differentiable Logic Gate Networks](https://mlsanity.com/paper/2210.08277/) (2022)
- [LogicNets: Co-Designed Neural Networks and Circuits for Extreme-Throughput Applications](https://mlsanity.com/paper/2004.03021/) (2020)
- [Convolutional Differentiable Logic Gate Networks](https://mlsanity.com/paper/2411.04732/) (2024)
- [PolyLUT: Learning Piecewise Polynomials for Ultra-Low Latency FPGA LUT-based Inference](https://mlsanity.com/paper/2309.02334/) (2023)
- [Differentiable Weightless Neural Networks](https://mlsanity.com/paper/2410.11112/) (2024)
- [NeuraLUT-Assemble: Hardware-aware Assembling of Sub-Neural Networks for Efficient LUT Inference](https://mlsanity.com/paper/2504.00592/) (2025)
- [Recurrent Deep Differentiable Logic Gate Networks](https://mlsanity.com/paper/2508.06097/) (2025)
- [PolyLUT: Ultra-low Latency Polynomial Inference with Hardware-Aware Structured Pruning](https://mlsanity.com/paper/2501.08043/) (2025)

## Built on by (most cited first, 9 of 9)

- [WARP Logic Neural Networks](https://mlsanity.com/paper/2602.03527/) (2026)
- [Learning Compact Boolean Networks](https://mlsanity.com/paper/2602.05830/) (2026)
- [BitLogic: Training Framework for Gradient-Based FPGA-Native Neural Networks](https://mlsanity.com/paper/2602.07400/) (2026)
- [From Arithmetic to Logic: The Resilience of Logic and Lookup-Based Neural Networks Under Parameter Bit-Flips](https://mlsanity.com/paper/2603.22770/) (2026)
- [Fitting Multilinear Polynomials for Logic Gate Networks](https://mlsanity.com/paper/2605.08657/) (2026)
- [Differentiable Logic Gate Networks for Low-Latency EEG Classification on Edge Devices](https://mlsanity.com/paper/2607.18149/) (2026)
- [DiffLUT-Net: Differentiable Training of FPGA LUT Networks with Learnable Connectivity](https://mlsanity.com/paper/2609.09254/) (2026)
- [Task-Aware Discretization of Differentiable Logic Gate Networks](https://mlsanity.com/paper/2609.33747/) (2026)
- [Overcoming Kernel Redundancy for Scaling Logic Gate Networks](https://mlsanity.com/paper/2610.01069/) (2026)

## Abstract (arXiv)

Differentiable logic gate networks (DLGNs) exhibit extraordinary efficiency at inference while sustaining competitive accuracy. But vanishing gradients, discretization errors, and high training cost impede scaling these networks. Even with dedicated parameter initialization schemes from subsequent works, increasing depth still harms accuracy. We show that the root cause of these issues lies in the underlying parametrization of logic gate neurons themselves. To overcome this issue, we propose a reparametrization that also shrinks the parameter size logarithmically in the number of inputs per gate. For binary inputs, this already reduces the model size by 4x, speeds up the backward pass by up to 1.86x, and converges in 8.5x fewer training steps. On top of that, we show that the accuracy on CIFAR-100 remains stable and sometimes superior to the original parametrization.