# Backpropagation as a Nilpotent Linear System

arXiv: https://arxiv.org/abs/2607.11289 · HTML: https://mlsanity.com/paper/2607.11289/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 10 of its 25 references that are in the map.
- First posted on arXiv 13 Jul 2026.

## Builds on (most cited first, 10 of 10)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Automatic differentiation in machine learning: a survey](https://mlsanity.com/paper/1502.05767/) (2015)
- [Deep Equilibrium Models](https://mlsanity.com/paper/1909.01377/) (2019)
- [A Physical Theory of Backpropagation: Exact Gradients from the Least-Action Principle](https://mlsanity.com/paper/2602.02281/) (2026)
- [A Two-Step Rule for Backpropagation](https://mlsanity.com/paper/2304.13537/) (2023)
- [Backpropagation and F-adjoint](https://mlsanity.com/paper/2304.13820/) (2023)
- [Learning by the F-adjoint](https://mlsanity.com/paper/2407.11049/) (2024)

## Abstract (arXiv)

Backpropagation is the computational engine of deep learning, yet its mathematical structure is typically treated as a procedural traversal of computational graphs. We present a global operator theory of the F-adjoint framework, which reformulates the layerwise backward recursion of an L-depth feedforward network into a single linear system (I-cB)Xs=bG, where bG is a source vector. We prove that the global backward operator cB is strictly block upper-triangular and nilpotent of index at most L. This nilpotency guarantees the exact termination of the Neumann series solution after at most L terms, revealing classical backpropagation to be mathematically equivalent to block back-substitution on an upper bidiagonal system. We formalise F-symmetry -- the condition in which the backward pass perfectly mirrors the forward pass -- identifying orthogonal weight matrices as canonical examples. Through worked numerical examples, we demonstrate how this operator perspective exposes the single-path collapse of strictly feedforward networks and its breakdown in residual architectures. Finally, we leverage this compositional structure to rigorously derive the mechanics of residual networks (gradient highways) and transfer learning (gradient truncation). This framework elevates backpropagation from an algorithmic recipe to a global nilpotent-operator formulation.