# Convex Geometry of ReLU-layers, Injectivity on the Ball and Local Reconstruction

arXiv: https://arxiv.org/abs/2307.09672 · HTML: https://mlsanity.com/paper/2307.09672/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 21 references that are in the map.
- First posted on arXiv 18 Jul 2023.

## Builds on (most cited first, 8 of 8)

- [Density estimation using Real NVP](https://mlsanity.com/paper/1605.08803/) (2016)
- [Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networks](https://mlsanity.com/paper/1602.07868/) (2016)
- [Adversarial Feature Learning](https://mlsanity.com/paper/1605.09782/) (2016)
- [Normalization Techniques in Training DNNs: Methodology, Analysis and Application](https://mlsanity.com/paper/2009.12836/) (2020)
- [Globally Injective ReLU Networks](https://mlsanity.com/paper/2006.08464/) (2020)
- [Signal Recovery from Pooling Representations](https://mlsanity.com/paper/1311.4025/) (2013)
- [Singular Values for ReLU Layers](https://mlsanity.com/paper/1812.02566/) (2018)
- [Analysis of Invariance and Robustness via Invertibility of ReLU-Networks](https://mlsanity.com/paper/1806.09730/) (2018)

## Built on by (most cited first, 2 of 2)

- [Complexity of Injectivity and Verification of ReLU Neural Networks](https://mlsanity.com/paper/2405.19805/) (2024)
- [Injectivity of ReLU-layers: Tools from Frame Theory](https://mlsanity.com/paper/2406.15856/) (2024)

## Abstract (arXiv)

The paper uses a frame-theoretic setting to study the injectivity of a ReLU-layer on the closed ball of ℝn and its non-negative part. In particular, the interplay between the radius of the ball and the bias vector is emphasized. Together with a perspective from convex geometry, this leads to a computationally feasible method of verifying the injectivity of a ReLU-layer under reasonable restrictions in terms of an upper bound of the bias vector. Explicit reconstruction formulas are provided, inspired by the duality concept from frame theory. All this gives rise to the possibility of quantifying the invertibility of a ReLU-layer and a concrete reconstruction algorithm for any input vector on the ball.