mlsanityData to 14 Oct 2026

Weaves, Wires, and Morphisms: Formalizing and Implementing the Algebra of Deep Learning

arXiv 2604.07242 ↗First posted 8 Apr 2026cs.LG math.CTcited by 0

Not on the map. Why?

Abstract (arXiv)

Despite deep learning models running well-defined mathematical functions, we lack a formal mathematical framework for describing model architectures. Ad-hoc notation, diagrams, and pseudocode poorly handle nonlinear broadcasting and the relationship between individual components and composed models. This paper introduces a categorical framework for deep learning models that formalizes broadcasting through the novel axis-stride and array-broadcasted categories. This allows the mathematical function underlying architectures to be precisely expressed and manipulated in a compositional manner. These mathematical definitions are translated into human manageable diagrams and machine manageable data structures. We provide a mirrored implementation in Python (pyncd) and TypeScript (tsncd) to show the universal aspect of our framework, along with features including algebraic construction, graph conversion, PyTorch compilation and diagram rendering. This lays the foundation for a systematic, formal approach to deep learning model design and analysis.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 23 of its 36 references that are in the map. First posted on arXiv 8 Apr 2026.

Builds on

23 of 36 references.

References · 23

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2017Graph Attention Networkscited by 8,923
  4. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
  5. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,598
  6. 2020GLU Variants Improve Transformercited by 1,804
  7. 2021Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gaugescited by 1,167
  8. 2024FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precisioncited by 519
  9. 2024AI and Memory Wallcited by 92
  10. 2022Formal Algorithms for Transformerscited by 72
  11. 2026FlashAttention-4: Algorithm and Kernel Pipelining Co-Design for Asymmetric Hardware Scalingcited by 66
  12. 2017Backprop as Functor: A compositional perspective on supervised learningcited by 44
11 more
  1. 2021Categorical Foundations of Gradient-Based Learningcited by 44
  2. 2021Category Theory in Machine Learningcited by 36
  3. 2024Position: Categorical Deep Learning is an Algebraic Theory of All Architecturescited by 32
  4. 2020Representable Markov Categories and Comparison of Statistical Experiments in Categorical Probabilitycited by 11
  5. 2021Named Tensor Notationcited by 11
  6. 2023An Introduction to String Diagrams for Computer Scientistscited by 9
  7. 2024Neural Circuit Diagrams: Robust Diagrams for the Communication, Implementation, and Analysis of Deep Learning Architecturescited by 8
  8. 2024FlashAttention on a Napkin: A Diagrammatic Approach to Deep Learning IO-Awarenesscited by 7
  9. 2024Fundamental Components of Deep Learning: A category-theoretic approachcited by 5
  10. 2024Deep Learning with Parametric Lensescited by 3
  11. 2025Accelerating Machine Learning Systems via Category Theory: Applications to Spherical Attention for Gene Regulatory Networkscited by 1

Built on by

Not cited yet.