# Learning Invariances with Generalised Input-Convex Neural Networks

arXiv: https://arxiv.org/abs/2204.07009 · HTML: https://mlsanity.com/paper/2204.07009/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 41 references that are in the map.
- First posted on arXiv 14 Apr 2022.

## Builds on (most cited first, 23 of 23)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Convex optimization](https://mlsanity.com/paper/2106.01946/) (2021)
- [Improved Training of Wasserstein GANs](https://mlsanity.com/paper/1704.00028/) (2017)
- [Spectral Normalization for Generative Adversarial Networks](https://mlsanity.com/paper/1802.05957/) (2018)
- [Normalizing Flows for Probabilistic Modeling and Inference](https://mlsanity.com/paper/1912.02762/) (2019)
- [Input Convex Neural Networks](https://mlsanity.com/paper/1609.07152/) (2016)
- [MADE: Masked Autoencoder for Distribution Estimation](https://mlsanity.com/paper/1502.03509/) (2015)
- [Fader Networks: Manipulating Images by Sliding Attributes](https://mlsanity.com/paper/1706.00409/) (2017)
- [Multi-Level Variational Autoencoder: Learning Disentangled Representations from Grouped Observations](https://mlsanity.com/paper/1705.08841/) (2017)
- [On the regularization of Wasserstein GANs](https://mlsanity.com/paper/1709.08894/) (2017)
- [Learning Latent Subspaces in Variational Autoencoders](https://mlsanity.com/paper/1812.06190/) (2018)
- [Disentangling Factors of Variation with Cycle-Consistent Variational Auto-Encoders](https://mlsanity.com/paper/1804.10469/) (2018)
- [Stochastic Backpropagation and Approximate Inference in Deep Generative Models](https://mlsanity.com/paper/1401.4082/) (2014)
- [3DMolNet: A Generative Network for Molecular Structures](https://mlsanity.com/paper/2010.06477/) (2020)
- [Learning Extremal Representations with Deep Archetypal Analysis](https://mlsanity.com/paper/2002.00815/) (2020)
- [Adversarial Information Factorization](https://mlsanity.com/paper/1711.05175/) (2017)
- [Input Invex Neural Network](https://mlsanity.com/paper/2106.08748/) (2021)
- [Inverse Learning of Symmetries](https://mlsanity.com/paper/2002.02782/) (2020)
- [Learning Conditional Invariance through Cycle Consistency](https://mlsanity.com/paper/2111.13185/) (2021)

## Built on by (most cited first, 4 of 4)

- [Principled Weight Initialisation for Input-Convex Neural Networks](https://mlsanity.com/paper/2312.12474/) (2023)
- [Asymmetric Certified Robustness via Feature-Convex Neural Networks](https://mlsanity.com/paper/2302.01961/) (2023)
- [Hyper Input Convex Neural Networks for Shape Constrained Learning and Optimal Transport](https://mlsanity.com/paper/2604.26942/) (2026)
- [Metric as Transform: Exploring beyond Affine Transform for Interpretable Neural Network](https://mlsanity.com/paper/2410.16159/) (2024)

## Abstract (arXiv)

Considering smooth mappings from input vectors to continuous targets, our goal is to characterise subspaces of the input domain, which are invariant under such mappings. Thus, we want to characterise manifolds implicitly defined by level sets. Specifically, this characterisation should be of a global parametric form, which is especially useful for different informed data exploration tasks, such as building grid-based approximations, sampling points along the level curves, or finding trajectories on the manifold. However, global parameterisations can only exist if the level sets are connected. For this purpose, we introduce a novel and flexible class of neural networks that generalise input-convex networks. These networks represent functions that are guaranteed to have connected level sets forming smooth manifolds on the input space. We further show that global parameterisations of these level sets can be always found efficiently. Lastly, we demonstrate that our novel technique for characterising invariances is a powerful generative data exploration tool in real-world applications, such as computational chemistry.