# Neural Networks as Local-to-Global Computations

arXiv: https://arxiv.org/abs/2603.14831 · HTML: https://mlsanity.com/paper/2603.14831/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 32 references that are in the map.
- First posted on arXiv 16 Mar 2026.

## Builds on (most cited first, 15 of 15)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [On the Number of Linear Regions of Deep Neural Networks](https://mlsanity.com/paper/1402.1869/) (2014)
- [Equilibrium Propagation: Bridging the Gap Between Energy-Based Models and Backpropagation](https://mlsanity.com/paper/1602.05179/) (2016)
- [Neural Sheaf Diffusion: A Topological Perspective on Heterophily and Oversmoothing in GNNs](https://mlsanity.com/paper/2202.04579/) (2022)
- [Predictive Coding Approximates Backprop along Arbitrary Computation Graphs](https://mlsanity.com/paper/2006.04182/) (2020)
- [Sheaf Neural Networks](https://mlsanity.com/paper/2012.06333/) (2020)
- [Sheaf Neural Networks with Connection Laplacians](https://mlsanity.com/paper/2206.08702/) (2022)
- [Joint Diffusion Processes as an Inductive Bias in Sheaf Neural Networks](https://mlsanity.com/paper/2407.20597/) (2024)
- [Sheaf Cohomology of Linear Predictive Coding Networks](https://mlsanity.com/paper/2511.11092/) (2025)
- [Topological Signatures of ReLU Neural Network Activation Patterns](https://mlsanity.com/paper/2510.12700/) (2025)
- [Selective Adaptation of Beliefs and Communication on Cellular Sheaves](https://mlsanity.com/paper/2601.22431/) (2026)

## Built on by (most cited first, 2 of 2)

- [Augmented Lagrangian Predictive Coding](https://mlsanity.com/paper/2605.31022/) (2026)
- [The Cognitive Categorical Transformer: Category-Theoretic Inductive Biases for Language Modeling](https://mlsanity.com/paper/2605.28864/) (2026)

## Abstract (arXiv)

We construct a cellular sheaf from any feedforward ReLU neural network by placing one vertex for each intermediate quantity in the forward pass and encoding each computational step - affine transformation, activation, output - as a restriction map on an edge. The restricted coboundary operator on the free coordinates is unitriangular, so its determinant is 1 and the restricted Laplacian is positive definite for every activation pattern. It follows that the relative cohomology vanishes and the forward pass output is the unique harmonic extension of the boundary data. The sheaf heat equation converges exponentially to this output despite the state-dependent switching introduced by piecewise linear activations. Unlike the forward pass, the heat equation propagates information bidirectionally across layers, enabling pinned neurons that impose constraints in both directions, training through local discrepancy minimization without a backward pass, and per-edge diagnostics that decompose network behavior by layer and operation type. We validate the framework experimentally on small synthetic tasks, confirming the convergence theorems and demonstrating that sheaf-based training, while not yet competitive with stochastic gradient descent, obeys quantitative scaling laws predicted by the theory.