mlsanityData to 7 Oct 2026

Deep Learning with Parametric Lenses

arXiv 2404.00408 ↗First posted 30 Mar 2024cs.LG cs.LOcited by 3

Not on the map. Why?

Abstract (arXiv)

We propose a categorical semantics for machine learning algorithms in terms of lenses, parametric maps, and reverse derivative categories. This foundation provides a powerful explanatory and unifying framework: it encompasses a variety of gradient descent algorithms such as ADAM, AdaGrad, and Nesterov momentum, as well as a variety of loss functions such as MSE and Softmax cross-entropy, and different architectures, shedding new light on their similarities and differences. Furthermore, our approach to learning has examples generalising beyond the familiar continuous domains (modelled in categories of smooth maps) and can be realised in the discrete setting of Boolean and polynomial circuits. We demonstrate the practical significance of our framework with an implementation in Python.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 27 of its 61 references that are in the map. First posted on arXiv 30 Mar 2024.

Builds on

27 of 61 references.

References · 27

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2022Generative Adversarial Networkscited by 5,603
  3. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,348
  4. 2021Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gaugescited by 1,131
  5. 2014Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Imagescited by 774
  6. 2016Learning to learn by gradient descent by gradient descentcited by 616
  7. 2015BinaryConnect: Training Deep Neural Networks with binary weights during propagationscited by 571
  8. 2014Understanding Deep Image Representations by Inverting Themcited by 399
  9. 2017Backprop as Functor: A compositional perspective on supervised learningcited by 44
  10. 2021Categorical Foundations of Gradient-Based Learningcited by 44
  11. 2023Architectures of Topological Deep Learning: A Survey of Message-Passing Topological Neural Networkscited by 43
  12. 2016Towards Verified Artificial Intelligencecited by 36
15 more
  1. 2018The simple essence of automatic differentiationcited by 22
  2. 2019Reverse derivative categoriescited by 15
  3. 2016Compositional game theorycited by 13
  4. 2019Lenses and Learnerscited by 11
  5. 2021Reverse Derivative Ascent: A Categorical Approach to Learning Boolean Circuitscited by 10
  6. 2023An Introduction to String Diagrams for Computer Scientistscited by 9
  7. 2019Compositional Deep Learningcited by 7
  8. 2020Profunctor Optics, a Categorical Updatecited by 6
  9. 2020Categorical Stochastic Processes and Likelihoodcited by 6
  10. 2019Differentiable Causal Computations via Delayed Tracecited by 5
  11. 2018Limits of bimorphic lensescited by 4
  12. 2022Categories of Differentiable Polynomial Circuits for Machine Learningcited by 4
  13. 2017Diagrammatic Semantics for Digital Circuitscited by 3
  14. 2021Translating Extensive Form Games to Open Games with Agencycited by 2
  15. 2019Generalized Lens Categories via functors Crm op→mathsfCatcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 2.

3 papers.

  1. 2026Self-Revising Discovery Systems for Science: A Categorical Framework for Agentic Artificial Intelligencecited by 3
  2. 2025Game-Theoretic Gradient Control for Robust Neural Network Trainingcited by 0
  3. 2026Weaves, Wires, and Morphisms: Formalizing and Implementing the Algebra of Deep Learningcited by 0