mlsanityData to 7 Oct 2026

Taming Binarized Neural Networks and Mixed-Integer Programs

arXiv 2310.04469 ↗First posted 5 Oct 2023cs.LG cs.AI math.OCcited by 4

Not on the map. Why?

Abstract (arXiv)

There has been a great deal of recent interest in binarized neural networks, especially because of their explainability. At the same time, automatic differentiation algorithms such as backpropagation fail for binarized neural networks, which limits their applicability. By reformulating the problem of training binarized neural networks as a subadditive dual of a mixed-integer program, we show that binarized neural networks admit a tame representation. This, in turn, makes it possible to use the framework of Bolte et al. for implicit differentiation, which offers the possibility for practical implementation of backpropagation in the context of binarized neural networks. This approach could also be used for a broader class of mixed-integer programs, beyond the training of binarized neural networks, as encountered in symbolic approaches to AI and beyond.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 20 of its 66 references that are in the map. First posted on arXiv 5 Oct 2023.

Builds on

20 of 66 references.

References · 20

  1. 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,031
  2. 2016XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networkscited by 829
  3. 2015BinaryConnect: Training Deep Neural Networks with binary weights during propagationscited by 571
  4. 2016DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradientscited by 557
  5. 2019Understanding Straight-Through Estimator in Training Activation Quantized Neural Netscited by 225
  6. 2017Towards Accurate Binary Convolutional Neural Networkcited by 148
  7. 2018Stochastic subgradient method converges on tame functionscited by 120
  8. 2017Deep Learning with Low Precision by Half-wave Gaussian Quantizationcited by 115
  9. 2019Conservative set valued fields, automatic differentiation, stochastic gradient method and deep learningcited by 71
  10. 2021A comprehensive review of Binary Neural Networkcited by 44
  11. 2021Nonsmooth Implicit Differentiation for Machine Learning and Optimizationcited by 43
  12. 2017Binarized Convolutional Landmark Localizers for Human Pose Estimation and Face Alignment with Limited Resourcescited by 23
8 more
  1. 2021Integer Programming for Causal Structure Learning in the Presence of Latent Variablescited by 14
  2. 2021Sub-bit Neural Networks: Learning to Compress and Accelerate Binary Neural Networkscited by 7
  3. 2023Global convergence of the gradient method for functions definable in o-minimal structurescited by 5
  4. 2020Symbolic Regression using Mixed-Integer Nonlinear Optimizationcited by 4
  5. 2020An Integer Programming Approach to Deep Neural Networks with Binary Activation Functionscited by 3
  6. 2021Learning Symbolic Expressions: Mixed-Integer Formulations, Cuts, and Heuristicscited by 3
  7. 2022Polynomial Matrix Inequalities within Tame Geometrycited by 2
  8. 2018On Subadditive Duality for Conic Mixed-Integer Programscited by 1

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 3.

4 papers.

  1. 2024Optimization Over Trained Neural Networks: Taking a Relaxing Walkcited by 10
  2. 2023Piecewise Polynomial Regression of Tame Functions via Integer Programmingcited by 1
  3. 2024Verifying Properties of Binary Neural Networks Using Sparse Polynomial Optimizationcited by 1
  4. 2024A constrained optimization approach to improve robustness of neural networkscited by 0