# Origami in N dimensions: How feed-forward networks manufacture linear separability

arXiv: https://arxiv.org/abs/2203.11355 · HTML: https://mlsanity.com/paper/2203.11355/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 51 references that are in the map.
- First posted on arXiv 21 Mar 2022.

## Builds on (most cited first, 25 of 25)

- [Deep Neural Networks as Gaussian Processes](https://mlsanity.com/paper/1711.00165/) (2017)
- [On the Expressive Power of Deep Neural Networks](https://mlsanity.com/paper/1606.05336/) (2016)
- [Understanding Deep Neural Networks with Rectified Linear Units](https://mlsanity.com/paper/1611.01491/) (2016)
- [Gradient Descent Happens in a Tiny Subspace](https://mlsanity.com/paper/1812.04754/) (2018)
- [The large learning rate phase of deep learning: the catapult mechanism](https://mlsanity.com/paper/2003.02218/) (2020)
- [Dynamical Isometry and a Mean Field Theory of CNNs: How to Train 10,000-Layer Vanilla Convolutional Neural Networks](https://mlsanity.com/paper/1806.05393/) (2018)
- [Deep ReLU Networks Have Surprisingly Few Activation Patterns](https://mlsanity.com/paper/1906.00904/) (2019)
- [Resurrecting the sigmoid in deep learning through dynamical isometry: theory and practice](https://mlsanity.com/paper/1711.04735/) (2017)
- [Tropical Geometry of Deep Neural Networks](https://mlsanity.com/paper/1805.07091/) (2018)
- [Reverse-Engineering Deep ReLU Networks](https://mlsanity.com/paper/1910.00744/) (2019)
- [Triple descent and the two kinds of overfitting: Where & why do they appear?](https://mlsanity.com/paper/2006.03509/) (2020)
- [Classification and Geometry of General Perceptual Manifolds](https://mlsanity.com/paper/1710.06487/) (2017)
- [Mad Max: Affine Spline Insights into Deep Learning](https://mlsanity.com/paper/1805.06576/) (2018)
- [The Geometry of Deep Networks: Power Diagram Subdivision](https://mlsanity.com/paper/1905.08443/) (2019)
- [Deep, Skinny Neural Networks are not Universal Approximators](https://mlsanity.com/paper/1810.00393/) (2018)
- [Dimensionality compression and expansion in Deep Neural Networks](https://mlsanity.com/paper/1906.00443/) (2019)
- [Empirical Studies on the Properties of Linear Regions in Deep Neural Networks](https://mlsanity.com/paper/2001.01072/) (2020)
- [Marvels and Pitfalls of the Langevin Algorithm in Noisy High-dimensional Inference](https://mlsanity.com/paper/1812.09066/) (2018)
- [Decomposing neural networks as mappings of correlation functions](https://mlsanity.com/paper/2202.04925/) (2022)
- [A singular Riemannian geometry approach to Deep Neural Networks I. Theoretical foundations](https://mlsanity.com/paper/2201.09656/) (2021)
- [A singular Riemannian geometry approach to Deep Neural Networks II. Reconstruction of 1-D equivalence classes](https://mlsanity.com/paper/2112.10583/) (2021)
- [Rademacher complexity and spin glasses: A link between the replica and statistical theories of learning](https://mlsanity.com/paper/1912.02729/) (2019)
- [Deep Autoencoders: From Understanding to Generalization Guarantees](https://mlsanity.com/paper/2009.09525/) (2020)
- [Geometry of Deep Convolutional Networks](https://mlsanity.com/paper/1905.08922/) (2019)
- [Disentangling deep neural networks with rectified linear units using duality](https://mlsanity.com/paper/2110.03403/) (2021)

## Built on by (most cited first, 9 of 9)

- [When Deep Learning Meets Polyhedral Theory: A Survey](https://mlsanity.com/paper/2305.00241/) (2023)
- [Interpreting Neural Networks through the Polytope Lens](https://mlsanity.com/paper/2211.12312/) (2022)
- [Neural population geometry and optimal coding of tasks with shared latent structure](https://mlsanity.com/paper/2402.16770/) (2024)
- [Orthogonal Representation Learning for Estimating Causal Quantities](https://mlsanity.com/paper/2502.04274/) (2025)
- [On Space Folds of ReLU Neural Networks](https://mlsanity.com/paper/2502.09954/) (2025)
- [A Perceptron-based Fine Approximation Technique for Linear Separation](https://mlsanity.com/paper/2309.06049/) (2023)
- [A singular Riemannian Geometry Approach to Deep Neural Networks III. Piecewise Differentiable Layers and Random Walks on n-dimensional Classes](https://mlsanity.com/paper/2404.06104/) (2024)
- [A unified theory of feature learning in RNNs and DNNs](https://mlsanity.com/paper/2602.15593/) (2026)
- [The Space Between: On Folding, Symmetries and Sampling](https://mlsanity.com/paper/2503.08502/) (2025)

## Abstract (arXiv)

Neural networks can implement arbitrary functions. But, mechanistically, what are the tools at their disposal to construct the target? For classification tasks, the network must transform the data classes into a linearly separable representation in the final hidden layer. We show that a feed-forward architecture has one primary tool at hand to achieve this separability: progressive folding of the data manifold in unoccupied higher dimensions. The operation of folding provides a useful intuition in low-dimensions that generalizes to high ones. We argue that an alternative method based on shear, requiring very deep architectures, plays only a small role in real-world networks. The folding operation, however, is powerful as long as layers are wider than the data dimensionality, allowing efficient solutions by providing access to arbitrary regions in the distribution, such as data points of one class forming islands within the other classes. We argue that a link exists between the universal approximation property in ReLU networks and the fold-and-cut theorem (Demaine et al., 1998) dealing with physical paper folding. Based on the mechanistic insight, we predict that the progressive generation of separability is necessarily accompanied by neurons showing mixed selectivity and bimodal tuning curves. This is validated in a network trained on the poker hand task, showing the emergence of bimodal tuning curves during training. We hope that our intuitive picture of the data transformation in deep networks can help to provide interpretability, and discuss possible applications to the theory of convolutional networks, loss landscapes, and generalization. TL;DR: Shows that the internal processing of deep networks can be thought of as literal folding operations on the data distribution in the N-dimensional activation space. A link to a well-known theorem in origami theory is provided.