# Expression of Fractals Through Neural Network Functions

arXiv: https://arxiv.org/abs/1905.11345 · HTML: https://mlsanity.com/paper/1905.11345/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 May 2019.

## Built on by (most cited first, 8 of 8)

- [When Deep Learning Meets Polyhedral Theory: A Survey](https://mlsanity.com/paper/2305.00241/) (2023)
- [Improving Fractal Pre-training](https://mlsanity.com/paper/2110.03091/) (2021)
- [Fractal Structure and Generalization Properties of Stochastic Optimization Algorithms](https://mlsanity.com/paper/2106.04881/) (2021)
- [Neural Network Approximation of Refinable Functions](https://mlsanity.com/paper/2107.13191/) (2021)
- [Fast Jacobian-Vector Product for Deep Networks](https://mlsanity.com/paper/2104.00219/) (2021)
- [Exact Loop Controllers for ReLU Realization of Homogeneous Curve Refinements](https://mlsanity.com/paper/2605.01655/) (2026)
- [Learning Fractals by Gradient Descent](https://mlsanity.com/paper/2303.12722/) (2023)
- [Exact ReLU realization of affine one-dimensional refinement iterates via residual memory and offset frames](https://mlsanity.com/paper/2607.20586/) (2026)

## Abstract (arXiv)

To help understand the underlying mechanisms of neural networks (NNs), several groups have, in recent years, studied the number of linear regions ℓ of piecewise linear functions generated by deep neural networks (DNN). In particular, they showed that ℓ can grow exponentially with the number of network parameters p, a property often used to explain the advantages of DNNs over shallow NNs in approximating complicated functions. Nonetheless, a simple dimension argument shows that DNNs cannot generate all piecewise linear functions with ℓ linear regions as soon as ℓ > p. It is thus natural to seek to characterize specific families of functions with ℓ linear regions that can be constructed by DNNs. Iterated Function Systems (IFS) generate sequences of piecewise linear functions Fk with a number of linear regions exponential in k. We show that, under mild assumptions, Fk can be generated by a NN using only O(k) parameters. IFS are used extensively to generate, at low computational cost, natural-looking landscape textures in artificial images. They have also been proposed for compression of natural images, albeit with less commercial success. The surprisingly good performance of this fractal-based compression suggests that our visual system may lock in, to some extent, on self-similarities in images. The combination of this phenomenon with the capacity, demonstrated here, of DNNs to efficiently approximate IFS may contribute to the success of DNNs, particularly striking for image processing tasks, as well as suggest new algorithms for representing self similarities in images based on the DNN mechanism.