mlsanityData to 14 Oct 2026

Linear Independence of Generalized Neurons and Related Functions

arXiv 2410.03693 ↗First posted 22 Sep 2024cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

The linear independence of neurons plays a significant role in theoretical analysis of neural networks. Specifically, given neurons H1, ..., Hn: bRN × bRd → bR, we are interested in the following question: when are H1(θ1, ·), ..., Hn(θn, ·) are linearly independent as the parameters θ1, ..., θn of these functions vary over bRN. Previous works give a complete characterization of two-layer neurons without bias, for generic smooth activation functions. In this paper, we study the problem for neurons with arbitrary layers and widths, giving a simple but complete characterization for generic analytic activation functions.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 3 of its 12 references that are in the map. First posted on arXiv 22 Sep 2024.

Builds on

3 of 12 references.

References · 3

  1. 2021Embedding Principle of Loss Landscape of Deep Neural Networkscited by 38
  2. 2023Geometry and Local Recovery of Global Minima of Two-layer Neural Networks at Overparameterizationcited by 4
  3. 2024Geometry of Critical Sets and Existence of Saddle Branches for Two-layer Neural Networkscited by 2

Built on by

Not cited yet.