# Taming Binarized Neural Networks and Mixed-Integer Programs

arXiv: https://arxiv.org/abs/2310.04469 · HTML: https://mlsanity.com/paper/2310.04469/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 66 references that are in the map.
- First posted on arXiv 5 Oct 2023.

## Builds on (most cited first, 20 of 20)

- [Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation](https://mlsanity.com/paper/1308.3432/) (2013)
- [XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks](https://mlsanity.com/paper/1603.05279/) (2016)
- [BinaryConnect: Training Deep Neural Networks with binary weights during propagations](https://mlsanity.com/paper/1511.00363/) (2015)
- [DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients](https://mlsanity.com/paper/1606.06160/) (2016)
- [Understanding Straight-Through Estimator in Training Activation Quantized Neural Nets](https://mlsanity.com/paper/1903.05662/) (2019)
- [Towards Accurate Binary Convolutional Neural Network](https://mlsanity.com/paper/1711.11294/) (2017)
- [Stochastic subgradient method converges on tame functions](https://mlsanity.com/paper/1804.07795/) (2018)
- [Deep Learning with Low Precision by Half-wave Gaussian Quantization](https://mlsanity.com/paper/1702.00953/) (2017)
- [Conservative set valued fields, automatic differentiation, stochastic gradient method and deep learning](https://mlsanity.com/paper/1909.10300/) (2019)
- [A comprehensive review of Binary Neural Network](https://mlsanity.com/paper/2110.06804/) (2021)
- [Nonsmooth Implicit Differentiation for Machine Learning and Optimization](https://mlsanity.com/paper/2106.04350/) (2021)
- [Binarized Convolutional Landmark Localizers for Human Pose Estimation and Face Alignment with Limited Resources](https://mlsanity.com/paper/1703.00862/) (2017)
- [Integer Programming for Causal Structure Learning in the Presence of Latent Variables](https://mlsanity.com/paper/2102.03129/) (2021)
- [Sub-bit Neural Networks: Learning to Compress and Accelerate Binary Neural Networks](https://mlsanity.com/paper/2110.09195/) (2021)
- [Symbolic Regression using Mixed-Integer Nonlinear Optimization](https://mlsanity.com/paper/2006.06813/) (2020)
- [Global convergence of the gradient method for functions definable in o-minimal structures](https://mlsanity.com/paper/2303.03534/) (2023)
- [An Integer Programming Approach to Deep Neural Networks with Binary Activation Functions](https://mlsanity.com/paper/2007.03326/) (2020)
- [Learning Symbolic Expressions: Mixed-Integer Formulations, Cuts, and Heuristics](https://mlsanity.com/paper/2102.08351/) (2021)
- [Polynomial Matrix Inequalities within Tame Geometry](https://mlsanity.com/paper/2206.03941/) (2022)
- [On Subadditive Duality for Conic Mixed-Integer Programs](https://mlsanity.com/paper/1808.10419/) (2018)

## Built on by (most cited first, 4 of 4)

- [Optimization Over Trained Neural Networks: Taking a Relaxing Walk](https://mlsanity.com/paper/2401.03451/) (2024)
- [Piecewise Polynomial Regression of Tame Functions via Integer Programming](https://mlsanity.com/paper/2311.13544/) (2023)
- [Verifying Properties of Binary Neural Networks Using Sparse Polynomial Optimization](https://mlsanity.com/paper/2405.17049/) (2024)
- [A constrained optimization approach to improve robustness of neural networks](https://mlsanity.com/paper/2409.13770/) (2024)

## Abstract (arXiv)

There has been a great deal of recent interest in binarized neural networks, especially because of their explainability. At the same time, automatic differentiation algorithms such as backpropagation fail for binarized neural networks, which limits their applicability. By reformulating the problem of training binarized neural networks as a subadditive dual of a mixed-integer program, we show that binarized neural networks admit a tame representation. This, in turn, makes it possible to use the framework of Bolte et al. for implicit differentiation, which offers the possibility for practical implementation of backpropagation in the context of binarized neural networks. This approach could also be used for a broader class of mixed-integer programs, beyond the training of binarized neural networks, as encountered in symbolic approaches to AI and beyond.