# On the Principles of ReLU Networks with One Hidden Layer

arXiv: https://arxiv.org/abs/2411.06728 · HTML: https://mlsanity.com/paper/2411.06728/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Nov 2024.

## Built on by (most cited first, 1 of 1)

- [On the Principles of Deep Feedforward ReLU Networks](https://mlsanity.com/paper/2607.07035/) (2026)

## Abstract (arXiv)

A neural network with one hidden layer or a two-layer network (regardless of the input layer) is the simplest feedforward neural network, whose mechanism may be the basis of more general network architectures. However, even to this type of simple architecture, it is also a ``black box''; that is, it remains unclear how to interpret the mechanism of its solutions obtained by the back-propagation algorithm and how to control the training process through a deterministic way. This paper systematically studies the first problem by constructing universal function-approximation solutions. It is shown that, both theoretically and experimentally, the training solution for the one-dimensional input could be completely understood, and that for a higher-dimensional input can also be well interpreted to some extent. Those results pave the way for thoroughly revealing the black box of two-layer ReLU networks and advance the understanding of deep ReLU networks.