# Linear Independence of Generalized Neurons and Related Functions

arXiv: https://arxiv.org/abs/2410.03693 · HTML: https://mlsanity.com/paper/2410.03693/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 3 of its 12 references that are in the map.
- First posted on arXiv 22 Sep 2024.

## Builds on (most cited first, 3 of 3)

- [Embedding Principle of Loss Landscape of Deep Neural Networks](https://mlsanity.com/paper/2105.14573/) (2021)
- [Geometry and Local Recovery of Global Minima of Two-layer Neural Networks at Overparameterization](https://mlsanity.com/paper/2309.00508/) (2023)
- [Geometry of Critical Sets and Existence of Saddle Branches for Two-layer Neural Networks](https://mlsanity.com/paper/2405.17501/) (2024)

## Abstract (arXiv)

The linear independence of neurons plays a significant role in theoretical analysis of neural networks. Specifically, given neurons H1, ..., Hn: bRN × bRd → bR, we are interested in the following question: when are H1(θ1, ·), ..., Hn(θn, ·) are linearly independent as the parameters θ1, ..., θn of these functions vary over bRN. Previous works give a complete characterization of two-layer neurons without bias, for generic smooth activation functions. In this paper, we study the problem for neurons with arbitrary layers and widths, giving a simple but complete characterization for generic analytic activation functions.