# A Note on the Representation Power of GHHs

arXiv: https://arxiv.org/abs/2101.11286 · HTML: https://mlsanity.com/paper/2101.11286/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 1 of its 10 references that are in the map.
- First posted on arXiv 27 Jan 2021.

## Builds on (most cited first, 1 of 1)

- [The Expressive Power of Neural Networks: A View from the Width](https://mlsanity.com/paper/1709.02540/) (2017)

## Built on by (most cited first, 3 of 3)

- [Towards Lower Bounds on the Depth of ReLU Neural Networks](https://mlsanity.com/paper/2105.14835/) (2021)
- [Minimum width for universal approximation using ReLU networks on compact domain](https://mlsanity.com/paper/2309.10402/) (2023)
- [Optimal bump functions for shallow ReLU networks: Weight decay, depth separation and the curse of dimensionality](https://mlsanity.com/paper/2209.01173/) (2022)

## Abstract (arXiv)

In this note we prove a sharp lower bound on the necessary number of nestings of nested absolute-value functions of generalized hinging hyperplanes (GHH) to represent arbitrary CPWL functions. Previous upper bound states that n+1 nestings is sufficient for GHH to achieve universal representation power, but the corresponding lower bound was unknown. We prove that n nestings is necessary for universal representation power, which provides an almost tight lower bound. We also show that one-hidden-layer neural networks don't have universal approximation power over the whole domain. The analysis is based on a key lemma showing that any finite sum of periodic functions is either non-integrable or the zero function, which might be of independent interest.