# Neural Networks Should Be Wide Enough to Learn Disconnected Decision Regions

arXiv: https://arxiv.org/abs/1803.00094 · HTML: https://mlsanity.com/paper/1803.00094/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Feb 2018.

## Built on by (most cited first, 17 of 17)

- [Towards Lower Bounds on the Depth of ReLU Neural Networks](https://mlsanity.com/paper/2105.14835/) (2021)
- [When Deep Learning Meets Polyhedral Theory: A Survey](https://mlsanity.com/paper/2305.00241/) (2023)
- [Lower Bounds on the Depth of Integral ReLU Neural Networks via Lattice Polytopes](https://mlsanity.com/paper/2302.12553/) (2023)
- [The Real Tropical Geometry of Neural Networks](https://mlsanity.com/paper/2403.11871/) (2024)
- [Training Fully Connected Neural Networks is existsℝ-Complete](https://mlsanity.com/paper/2204.01368/) (2022)
- [Activation degree thresholds and expressiveness of polynomial neural networks](https://mlsanity.com/paper/2408.04569/) (2024)
- [Minimum Width of Leaky-ReLU Neural Networks for Uniform Universal Approximation](https://mlsanity.com/paper/2305.18460/) (2023)
- [ReLU Neural Networks of Polynomial Size for Exact Maximum Flow Computation](https://mlsanity.com/paper/2102.06635/) (2021)
- [Parameter Symmetry Potentially Unifies Deep Learning Theory](https://mlsanity.com/paper/2502.05300/) (2025)
- [Topological Expressivity of ReLU Neural Networks](https://mlsanity.com/paper/2310.11130/) (2023)
- [A shallow neural model for relation prediction](https://mlsanity.com/paper/2101.09090/) (2021)
- [Topological Deep Learning: Classification Neural Networks](https://mlsanity.com/paper/2102.08354/) (2021)
- [A Critical Review of Information Bottleneck Theory and its Applications to Deep Learning](https://mlsanity.com/paper/2105.04405/) (2021)
- [On the Expressive Power of Tree-Structured Probabilistic Circuits](https://mlsanity.com/paper/2410.05465/) (2024)
- [Understanding Nonlinear Implicit Bias via Region Counts in Input Space](https://mlsanity.com/paper/2505.11370/) (2025)
- [Solving adversarial examples requires solving exponential misalignment](https://mlsanity.com/paper/2603.03507/) (2026)
- [Empirical Evidence for Simply Connected Decision Regions in Image Classifiers](https://mlsanity.com/paper/2605.06380/) (2026)

## Abstract (arXiv)

In the recent literature the important role of depth in deep learning has been emphasized. In this paper we argue that sufficient width of a feedforward network is equally important by answering the simple question under which conditions the decision regions of a neural network are connected. It turns out that for a class of activation functions including leaky ReLU, neural networks having a pyramidal structure, that is no layer has more hidden units than the input dimension, produce necessarily connected decision regions. This implies that a sufficiently wide hidden layer is necessary to guarantee that the network can produce disconnected decision regions. We discuss the implications of this result for the construction of neural networks, in particular the relation to the problem of adversarial manipulation of classifiers.