# Deep Differentiable Logic Gate Networks

arXiv: https://arxiv.org/abs/2210.08277 · HTML: https://mlsanity.com/paper/2210.08277/

## Key facts

- Cited by 67 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 79 references that are in the map.
- First posted on arXiv 15 Oct 2022.

## Builds on (most cited first, 30 of 33)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding](https://mlsanity.com/paper/1510.00149/) (2015)
- [The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables](https://mlsanity.com/paper/1611.00712/) (2016)
- [Learning Sparse Neural Networks through L0 Regularization](https://mlsanity.com/paper/1712.01312/) (2017)
- [What is the State of Neural Network Pruning?](https://mlsanity.com/paper/2003.03033/) (2020)
- [Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networks](https://mlsanity.com/paper/2102.00554/) (2021)
- [PACT: Parameterized Clipping Activation for Quantized Neural Networks](https://mlsanity.com/paper/1805.06085/) (2018)
- [Deep Learning with Limited Numerical Precision](https://mlsanity.com/paper/1502.02551/) (2015)
- [Scalable Training of Artificial Neural Networks with Adaptive Sparse Connectivity inspired by Network Science](https://mlsanity.com/paper/1707.04780/) (2017)
- [Soft-DTW: a Differentiable Loss Function for Time-Series](https://mlsanity.com/paper/1703.01541/) (2017)
- [Variational Dropout Sparsifies Deep Neural Networks](https://mlsanity.com/paper/1701.05369/) (2017)
- [Binary Neural Networks: A Survey](https://mlsanity.com/paper/2004.03333/) (2020)
- [FINN: A Framework for Fast, Scalable Binarized Neural Network Inference](https://mlsanity.com/paper/1612.07119/) (2016)
- [Differentiable Ranks and Sorting using Optimal Transport](https://mlsanity.com/paper/1905.11885/) (2019)
- [Analyzing Differentiable Fuzzy Logic Operators](https://mlsanity.com/paper/2002.06100/) (2020)
- [The Unreasonable Effectiveness of Random Pruning: Return of the Most Naive Baseline for Sparse Training](https://mlsanity.com/paper/2202.02643/) (2022)
- [Machine Learning at the Network Edge: A Survey](https://mlsanity.com/paper/1908.00080/) (2019)
- [Learning with Differentiable Perturbed Optimizers](https://mlsanity.com/paper/2002.08676/) (2020)
- [Weight Agnostic Neural Networks](https://mlsanity.com/paper/1906.04358/) (2019)
- [Effective Sparsification of Neural Networks with Global Sparsity Constraint](https://mlsanity.com/paper/2105.01571/) (2021)
- [PruneTrain: Fast Neural Network Training by Dynamic Sparse Model Reconfiguration](https://mlsanity.com/paper/1901.09290/) (2019)
- [Differentiable Sorting Networks for Scalable Sorting and Ranking Supervision](https://mlsanity.com/paper/2105.04019/) (2021)
- [Monotonic Differentiable Sorting Networks](https://mlsanity.com/paper/2203.09630/) (2022)
- [Learning with Algorithmic Supervision via Continuous Relaxations](https://mlsanity.com/paper/2110.05651/) (2021)
- [Differentiable Logic Machines](https://mlsanity.com/paper/2102.11529/) (2021)
- [GenDR: A Generalized Differentiable Renderer](https://mlsanity.com/paper/2204.13845/) (2022)
- [Learning Binary Decision Trees by Argmin Differentiation](https://mlsanity.com/paper/2010.04627/) (2020)
- [Learning with Differentiable Algorithms](https://mlsanity.com/paper/2209.00616/) (2022)

## Built on by (most cited first, 30 of 67)

- [Interpretable Neural-Symbolic Concept Reasoning](https://mlsanity.com/paper/2304.14068/) (2023)
- [Learning Transformer Programs](https://mlsanity.com/paper/2306.01128/) (2023)
- [Convolutional Differentiable Logic Gate Networks](https://mlsanity.com/paper/2411.04732/) (2024)
- [Differentiable Weightless Neural Networks](https://mlsanity.com/paper/2410.11112/) (2024)
- [A Comprehensive Survey on Self-Interpretable Neural Networks](https://mlsanity.com/paper/2501.15638/) (2025)
- [Uncertainty Quantification via Stable Distribution Propagation](https://mlsanity.com/paper/2402.08324/) (2024)
- [Learning Interpretable Differentiable Logic Networks](https://mlsanity.com/paper/2407.04168/) (2024)
- [Differentiable Logic Cellular Automata: From Game of Life to Pattern Generation](https://mlsanity.com/paper/2506.04912/) (2025)
- [Light Differentiable Logic Gate Networks](https://mlsanity.com/paper/2510.03250/) (2025)
- [Self-Supervised Transformers as Iterative Solution Improvers for Constraint Satisfaction](https://mlsanity.com/paper/2502.15794/) (2025)
- [WARP-LUTs -- Walsh-Assisted Relaxation for Probabilistic Look Up Tables](https://mlsanity.com/paper/2510.15655/) (2025)
- [LILogic Net: Compact Logic Gate Networks with Learnable Connectivity for Efficient Hardware Deployment](https://mlsanity.com/paper/2511.12340/) (2025)
- [Improving Discrete Optimisation Via Decoupled Straight-Through Estimator](https://mlsanity.com/paper/2410.13331/) (2024)
- [A Method for Optimizing Connections in Differentiable Logic Gate Networks](https://mlsanity.com/paper/2507.06173/) (2025)
- [LogicCBMs: Logic-Enhanced Concept-Based Learning](https://mlsanity.com/paper/2512.07383/) (2025)
- [Logic Gate Neural Networks are Good for Verification](https://mlsanity.com/paper/2505.19932/) (2025)
- [NestQuant: Post-Training Integer-Nesting Quantization for On-Device DNN](https://mlsanity.com/paper/2506.17870/) (2025)
- [Scalable Interconnect Learning in Boolean Networks](https://mlsanity.com/paper/2507.02585/) (2025)
- [A Comparative Study of Neurosymbolic AI Approaches to Interpretable Logical Reasoning](https://mlsanity.com/paper/2508.03366/) (2025)
- [BOLD: Boolean Logic Deep Learning](https://mlsanity.com/paper/2405.16339/) (2024)
- [Towards Narrowing the Generalization Gap in Deep Boolean Networks](https://mlsanity.com/paper/2409.05905/) (2024)
- [eXpLogic: Explaining Logic Types and Patterns in DiffLogic Networks](https://mlsanity.com/paper/2503.09910/) (2025)
- [LL-ViT: Edge Deployable Vision Transformers with Look Up Table Neurons](https://mlsanity.com/paper/2511.00812/) (2025)
- [Explicit Fuzzy Logic in the Feed-Forward Layer: Self-Forgetting Quantifiers Discover Legible Grammatical-Licensing Detectors](https://mlsanity.com/paper/2606.31845/) (2026)
- [Legible-by-Construction: Attention and End-to-End Transformers](https://mlsanity.com/paper/2607.04319/) (2026)
- [Compact Rule-Based Classifier Learning via Gradient Descent](https://mlsanity.com/paper/2502.01375/) (2025)
- [Learning Interpretable Differentiable Logic Networks for Tabular Regression](https://mlsanity.com/paper/2505.23615/) (2025)
- [WARP Logic Neural Networks](https://mlsanity.com/paper/2602.03527/) (2026)
- [Learning with Boolean threshold functions](https://mlsanity.com/paper/2602.17493/) (2026)
- [Polynomial Surrogate Training for Differentiable Ternary Logic Gate Networks](https://mlsanity.com/paper/2603.00302/) (2026)

## Abstract (arXiv)

Recently, research has increasingly focused on developing efficient neural network architectures. In this work, we explore logic gate networks for machine learning tasks by learning combinations of logic gates. These networks comprise logic gates such as "AND" and "XOR", which allow for very fast execution. The difficulty in learning logic gate networks is that they are conventionally non-differentiable and therefore do not allow training with gradient descent. Thus, to allow for effective training, we propose differentiable logic gate networks, an architecture that combines real-valued logics and a continuously parameterized relaxation of the network. The resulting discretized logic gate networks achieve fast inference speeds, e.g., beyond a million images of MNIST per second on a single CPU core.