# CDiNN -Convex Difference Neural Networks

arXiv: https://arxiv.org/abs/2103.17231 · HTML: https://mlsanity.com/paper/2103.17231/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 22 references that are in the map.
- First posted on arXiv 31 Mar 2021.

## Builds on (most cited first, 8 of 8)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [Input Convex Neural Networks](https://mlsanity.com/paper/1609.07152/) (2016)
- [Optimal Control Via Neural Networks: A Convex Approach](https://mlsanity.com/paper/1805.11835/) (2018)
- [Neural Network with Unbounded Activation Functions is Universal Approximator](https://mlsanity.com/paper/1505.03654/) (2015)
- [Input Convex Neural Networks for Building MPC](https://mlsanity.com/paper/2011.13227/) (2020)
- [The Curious Case of Convex Neural Networks](https://mlsanity.com/paper/2006.05103/) (2020)
- [Input Convex Neural Networks for Optimal Voltage Regulation](https://mlsanity.com/paper/2002.08684/) (2020)

## Built on by (most cited first, 6 of 6)

- [Principled Weight Initialisation for Input-Convex Neural Networks](https://mlsanity.com/paper/2312.12474/) (2023)
- [Neural Networks and (Virtual) Extended Formulations](https://mlsanity.com/paper/2411.03006/) (2024)
- [Defining Neural Network Architecture through Polytope Structures of Dataset](https://mlsanity.com/paper/2402.02407/) (2024)
- [Differentiable neural network representation of multi-well, locally-convex potentials](https://mlsanity.com/paper/2506.17242/) (2025)
- [Hidden Monotonicity: Explaining Deep Neural Networks via their DC Decomposition](https://mlsanity.com/paper/2601.07700/) (2026)
- [An Improved Boosted DC Algorithm for Nonsmooth Functions with Applications in Image Recovery](https://mlsanity.com/paper/2602.04237/) (2026)

## Abstract (arXiv)

Neural networks with ReLU activation function have been shown to be universal function approximators and learn function mapping as non-smooth functions. Recently, there is considerable interest in the use of neural networks in applications such as optimal control. It is well-known that optimization involving non-convex, non-smooth functions are computationally intensive and have limited convergence guarantees. Moreover, the choice of optimization hyper-parameters used in gradient descent/ascent significantly affect the quality of the obtained solutions. A new neural network architecture called the Input Convex Neural Networks (ICNNs) learn the output as a convex function of inputs thereby allowing the use of efficient convex optimization methods. Use of ICNNs for determining the input for minimizing output has two major problems: learning of a non-convex function as a convex mapping could result in significant function approximation error, and we also note that the existing representations cannot capture simple dynamic structures like linear time delay systems. We attempt to address the above problems by introduction of a new neural network architecture, which we call the CDiNN, which learns the function as a difference of polyhedral convex functions from data. We also discuss that, in some cases, the optimal input can be obtained from CDiNN through difference of convex optimization with convergence guarantees and that at each iteration, the problem is reduced to a linear programming problem.