# Approximating Lipschitz continuous functions with GroupSort neural networks

arXiv: https://arxiv.org/abs/2006.05254 · HTML: https://mlsanity.com/paper/2006.05254/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Jun 2020.

## Built on by (most cited first, 27 of 27)

- [Generative Modeling with Optimal Transport Maps](https://mlsanity.com/paper/2110.02999/) (2021)
- [Impossibility Theorems for Feature Attribution](https://mlsanity.com/paper/2212.11870/) (2022)
- [Rethinking Lipschitz Neural Networks and Certified Robustness: A Boolean Function Perspective](https://mlsanity.com/paper/2210.01787/) (2022)
- [Pay attention to your loss: understanding misconceptions about 1-Lipschitz neural networks](https://mlsanity.com/paper/2104.05097/) (2021)
- [Learning by Transference: Training Graph Neural Networks on Growing Graphs](https://mlsanity.com/paper/2106.03693/) (2021)
- [On the Number of Regions of Piecewise Linear Neural Networks](https://mlsanity.com/paper/2206.08615/) (2022)
- [Wasserstein Generative Adversarial Uncertainty Quantification in Physics-Informed Neural Networks](https://mlsanity.com/paper/2108.13054/) (2021)
- [Lyapunov-Net: A Deep Neural Network Architecture for Lyapunov Function Approximation](https://mlsanity.com/paper/2109.13359/) (2021)
- [Approximation of Lipschitz Functions using Deep Spline Neural Networks](https://mlsanity.com/paper/2204.06233/) (2022)
- [DP-SGD Without Clipping: The Lipschitz Neural Network Way](https://mlsanity.com/paper/2305.16202/) (2023)
- [Robust One-Class Classification with Signed Distance Function using 1-Lipschitz Neural Networks](https://mlsanity.com/paper/2303.01978/) (2023)
- [Universal Approximation of Residual Flows in Maximum Mean Discrepancy](https://mlsanity.com/paper/2103.05793/) (2021)
- [Novel Quadratic Constraints for Extending LipSDP beyond Slope-Restricted Activations](https://mlsanity.com/paper/2401.14033/) (2024)
- [Convolutional Neural Networks as 2-D systems](https://mlsanity.com/paper/2303.03042/) (2023)
- [GAN Estimation of Lipschitz Optimal Transport Maps](https://mlsanity.com/paper/2202.07965/) (2022)
- [Fast and scalable Wasserstein-1 neural optimal transport solver for single-cell perturbation prediction](https://mlsanity.com/paper/2411.00614/) (2024)
- [Wasserstein Adversarially Regularized Graph Autoencoder](https://mlsanity.com/paper/2111.04981/) (2021)
- [A Complete Set of Quadratic Constraints for Repeated ReLU and Generalizations](https://mlsanity.com/paper/2407.06888/) (2024)
- [Towards Better Multi-head Attention via Channel-wise Sample Permutation](https://mlsanity.com/paper/2410.10914/) (2024)
- [On the Existence of Universal Simulators of Attention](https://mlsanity.com/paper/2506.18739/) (2025)
- [ExLipBaB: Exact Lipschitz Constant Computation for Piecewise Linear Neural Networks](https://mlsanity.com/paper/2602.15499/) (2026)
- [Approximating Probability Distributions by using Wasserstein Generative Adversarial Networks](https://mlsanity.com/paper/2103.10060/) (2021)
- [Active Learning for Regression based on Wasserstein distance and GroupSort Neural Networks](https://mlsanity.com/paper/2403.15108/) (2024)
- [Modeling Sampling Distributions of Test Statistics with Autograd](https://mlsanity.com/paper/2405.02488/) (2024)
- [Neural Networks With Dense Weights Are Not Universal Approximators](https://mlsanity.com/paper/2602.07618/) (2026)
- [Self-Review Reinforcement Learning (SRRL) with Cross-Episode Memory and Policy Distillation](https://mlsanity.com/paper/2607.05541/) (2026)
- [Mutual Equilibrium: Multimodal Representation Learning through Reciprocal Feedback](https://mlsanity.com/paper/2609.39456/) (2026)

## Abstract (arXiv)

Recent advances in adversarial attacks and Wasserstein GANs have advocated for use of neural networks with restricted Lipschitz constants. Motivated by these observations, we study the recently introduced GroupSort neural networks, with constraints on the weights, and make a theoretical step towards a better understanding of their expressive power. We show in particular how these networks can represent any Lipschitz continuous piecewise linear functions. We also prove that they are well-suited for approximating Lipschitz continuous functions and exhibit upper bounds on both the depth and size. To conclude, the efficiency of GroupSort networks compared with more standard ReLU networks is illustrated in a set of synthetic experiments.