mlsanityData to 7 Oct 2026

Lossless Compression of Deep Neural Networks

arXiv 2001.00218 ↗First posted 1 Jan 2020cs.LG cs.DS math.OCcited by 34

Not on the map. Why?

Abstract (arXiv)

Deep neural networks have been successful in many predictive modeling tasks, such as image and language recognition, where large neural networks are often used to obtain good accuracy. Consequently, it is challenging to deploy these networks under limited computational resources, such as in mobile devices. In this work, we introduce an algorithm that removes units and layers of a neural network while not changing the output that is produced, which thus implies a lossless compression. This algorithm, which we denote as LEO (Lossless Expressiveness Optimization), relies on Mixed-Integer Linear Programming (MILP) to identify Rectified Linear Units (ReLUs) with linear behavior over the input domain. By using L1 regularization to induce such behavior, we can benefit from training over a larger architecture than we would later use in the environment where the trained neural network is deployed.

Key facts. Cited by 34 papers in the map (data through 7 Oct 2026). First posted on arXiv 1 Jan 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 9; 2022: 13; 2023: 6; 2024: 3; 2025: 2; 2026: 1.

34 papers.

Sort:

  1. 2021Towards Lower Bounds on the Depth of ReLU Neural Networkscited by 55
  2. 2022The Combinatorial Brain Surgeon: Pruning Weights That Cancel One Another in Neural Networkscited by 41
  3. 2021Partition-based formulations for mixed-integer optimization of trained ReLU neural networkscited by 39
  4. 2023When Deep Learning Meets Polyhedral Theory: A Surveycited by 39
  5. 2022OMLT: Optimization & Machine Learning Toolkitcited by 34
  6. 2022Pruning's Effect on Generalization Through the Lens of Training and Regularizationcited by 31
  7. 2021Parameter identifiability of a deep feedforward ReLU neural networkcited by 24
  8. 2021Scaling Up Exact Neural Network Compression by ReLU Stabilitycited by 19
  9. 2021On the Expected Complexity of Maxout Networkscited by 15
  10. 2021Between steps: Intermediate relaxations between big-M and convex hull formulationscited by 11
  11. 2022Optimizing Objective Functions from Trained ReLU Neural Networks via Samplingcited by 11
  12. 2024Optimization Over Trained Neural Networks: Taking a Relaxing Walkcited by 10
22 more
  1. 2022Recall Distortion in Neural Network Pruning and the Undecayed Pruning Algorithmcited by 9
  2. 2023Getting Away with More Network Pruning: From Sparsity to Geometry and Linear Regionscited by 8
  3. 2021Universal approximation and model compression for radial neural networkscited by 7
  4. 2022P-split formulations: A class of intermediate formulations between big-M and convex hull for disjunctive constraintscited by 7
  5. 2023Computational Tradeoffs of Optimization-Based Bound Tightening in ReLU Networkscited by 6
  6. 2022Deep Neural Networks pruning via the Structured Perspective Regularizationcited by 4
  7. 2022Quiver neural networkscited by 4
  8. 2024Free Lunch in the Forest: Functionally-Identical Pruning of Boosted Tree Ensemblescited by 4
  9. 2021On the Impact of Stable Ranks in Deep Netscited by 3
  10. 2022Training Thinner and Deeper Neural Networks: Jumpstart Regularizationcited by 3
  11. 2025Test-Time Steering for Lossless Text Compression via Weighted Product of Expertscited by 3
  12. 2022The Effect of Data Dimensionality on Neural Network Prunabilitycited by 2
  13. 2023Model-based feature selection for neural networks: A mixed-integer programming approachcited by 2
  14. 2021Optimizing over an ensemble of neural networkscited by 1
  15. 2022Support Vector Machines with the Hard-Margin Loss: Optimal Training via Combinatorial Benders' Cutscited by 1
  16. 2022Multi-Objective Linear Ensembles for Robust and Sparse Training of Few-Bit Neural Networkscited by 1
  17. 2023Learning Prescriptive ReLU Networkscited by 1
  18. 2022Towards Global Neural Network Abstractions with Locally-Exact Reconstructioncited by 0
  19. 2023Proximity to Losslessly Compressible Parameterscited by 0
  20. 2024Tightening convex relaxations of trained neural networks: a unified approach for convex and S-shaped activationscited by 0
  21. 2025Optimization over Trained (and Sparse) Neural Networks: A Surrogate within a Surrogatecited by 0
  22. 2026Fast SDP certification of neural networks : towards large multi-class datasetscited by 0