mlsanityData to 14 Oct 2026

From Complexity to Clarity: Analytical Expressions of Deep Neural Network Weights via Clifford's Geometric Algebra and Convexity

arXiv 2309.16512 ↗First posted 28 Sep 2023cs.LG cs.AI cs.NEcited by 4

Not on the map. Why?

Abstract (arXiv)

In this paper, we introduce a novel analysis of neural networks based on geometric (Clifford) algebra and convex optimization. We show that optimal weights of deep ReLU neural networks are given by the wedge product of training samples when trained with standard regularized loss. Furthermore, the training problem reduces to convex optimization over wedge product features, which encode the geometric structure of the training dataset. This structure is given in terms of signed volumes of triangles and parallelotopes generated by data vectors. The convex problem finds a small subset of samples via ℓ1 regularization to discover only relevant wedge product features. Our analysis provides a novel perspective on the inner workings of deep neural networks and sheds light on the role of the hidden layers.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 16 of its 33 references that are in the map. First posted on arXiv 28 Sep 2023.

Builds on

16 of 33 references.

References · 16

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,621
  3. 2021Convex optimizationcited by 2,515
  4. 2020Prevalence of Neural Collapse during the terminal phase of deep learning trainingcited by 719
  5. 2018On Lazy Training in Differentiable Programmingcited by 675
  6. 2020Neural Networks are Convex Regularizers: Exact Polynomial-time Convex Optimization Formulations for Two-layer Networkscited by 98
  7. 2020Revealing the Structure of Deep Neural Networks via Convex Dualitycited by 54
  8. 2020Convex Geometry and Duality of Over-parameterized Neural Networkscited by 49
  9. 2020Vector-output ReLU Neural Network Problems are Copositive Programs: Convex Analysis of Two Layer Networks and Polynomial-time Algorithmscited by 34
  10. 2020The Hidden Convex Optimization Landscape of Two-Layer ReLU Neural Networks: an Exact Characterization of the Optimal Solutionscited by 30
  11. 2021Global Optimality Beyond Two Layers: Training Deep ReLU Networks via Convex Programscited by 25
  12. 2021Demystifying Batch Normalization in ReLU Networks: Equivalent Convex Optimization Models and Implicit Regularizationcited by 21
4 more
  1. 2022Wide and Deep Neural Networks Achieve Optimality for Classificationcited by 12
  2. 2021The Convex Geometry of Backpropagation: Neural Network Gradient Flows Converge to Extreme Points of the Dual Convex Programcited by 11
  3. 2023Globally Optimal Training of Neural Networks with Threshold Activation Functionscited by 9
  4. 2021Training Quantized Neural Networks to Global Optimality via Semidefinite Programmingcited by 6

Built on by

Cited by year

Citing papers per year: 2024: 3; 2025: 1.

4 papers.

  1. 2025Composing Linear Layers from Irreduciblescited by 2
  2. 2024A Library of Mirrors: Deep Neural Nets in Low Dimensions are Convex Lasso Models with Reflection Featurescited by 1
  3. 2024Randomized Geometric Algebra Methods for Convex Neural Networkscited by 1
  4. 2024ConvexECG: Lightweight and Explainable Neural Networks for Personalized, Continuous Cardiac Monitoringcited by 0