# Gated Linear Networks

arXiv: https://arxiv.org/abs/1910.01526 · HTML: https://mlsanity.com/paper/1910.01526/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Sep 2019.

## Built on by (most cited first, 26 of 26)

- [Acquisition of Chess Knowledge in AlphaZero](https://mlsanity.com/paper/2111.09259/) (2021)
- [Scaling Forward Gradient With Local Losses](https://mlsanity.com/paper/2210.03310/) (2022)
- [MesaNet: Sequence Modeling by Locally Optimal Test-Time Training](https://mlsanity.com/paper/2506.05233/) (2025)
- [The Neural Race Reduction: Dynamics of Abstraction in Gated Networks](https://mlsanity.com/paper/2207.10430/) (2022)
- [Avoiding Catastrophe: Active Dendrites Enable Multi-Task Learning in Dynamic Environments](https://mlsanity.com/paper/2201.00042/) (2021)
- [The Influence of Learning Rule on Representation Dynamics in Wide Neural Networks](https://mlsanity.com/paper/2210.02157/) (2022)
- [Globally Gated Deep Linear Networks](https://mlsanity.com/paper/2210.17449/) (2022)
- [A foundational neural operator that continuously learns without forgetting](https://mlsanity.com/paper/2310.18885/) (2023)
- [Self-Activating Neural Ensembles for Continual Reinforcement Learning](https://mlsanity.com/paper/2301.00141/) (2022)
- [Levin Tree Search with Context Models](https://mlsanity.com/paper/2305.16945/) (2023)
- [Training Deep Architectures Without End-to-End Backpropagation: A Survey on the Provably Optimal Methods](https://mlsanity.com/paper/2101.03419/) (2021)
- [Local Learning with Neuron Groups](https://mlsanity.com/paper/2301.07635/) (2023)
- [The Implicit Bias of Gradient Descent on Generalized Gated Linear Networks](https://mlsanity.com/paper/2202.02649/) (2022)
- [Memorization and Knowledge Injection in Gated LLMs](https://mlsanity.com/paper/2504.21239/) (2025)
- [A statistical physics framework for optimal learning](https://mlsanity.com/paper/2507.07907/) (2025)
- [Adapting the Function Approximation Architecture in Online Reinforcement Learning](https://mlsanity.com/paper/2106.09776/) (2021)
- [Expressivity of Neural Networks with Random Weights and Learned Biases](https://mlsanity.com/paper/2407.00957/) (2024)
- [Gated Linear Model induced U-net for surrogate modeling and uncertainty quantification](https://mlsanity.com/paper/2111.05123/) (2021)
- [Explicitising The Implicit Intrepretability of Deep Neural Networks Via Duality](https://mlsanity.com/paper/2203.16455/) (2022)
- [Leader-Follower Neural Networks with Local Error Signals Inspired by Complex Collectives](https://mlsanity.com/paper/2310.07885/) (2023)
- [Geometry-Aware Probabilistic Circuits via Voronoi Tessellations](https://mlsanity.com/paper/2603.11946/) (2026)
- [DynaTab: Dynamic Feature Ordering as Neural Rewiring for High-Dimensional Tabular Data](https://mlsanity.com/paper/2605.03430/) (2026)
- [Reinforcement Learning with Information-Theoretic Actuation](https://mlsanity.com/paper/2109.15147/) (2021)
- [SPINE: Soft Piecewise Interpretable Neural Equations](https://mlsanity.com/paper/2111.10622/) (2021)
- [Hierarchical Partitioning Forecaster](https://mlsanity.com/paper/2305.13063/) (2023)
- [Decentralised AI Training and Inference with BlockTrain](https://mlsanity.com/paper/2606.24722/) (2026)

## Abstract (arXiv)

This paper presents a new family of backpropagation-free neural architectures, Gated Linear Networks (GLNs). What distinguishes GLNs from contemporary neural networks is the distributed and local nature of their credit assignment mechanism; each neuron directly predicts the target, forgoing the ability to learn feature representations in favor of rapid online learning. Individual neurons can model nonlinear functions via the use of data-dependent gating in conjunction with online convex optimization. We show that this architecture gives rise to universal learning capabilities in the limit, with effective model capacity increasing as a function of network size in a manner comparable with deep ReLU networks. Furthermore, we demonstrate that the GLN learning mechanism possesses extraordinary resilience to catastrophic forgetting, performing comparably to a MLP with dropout and Elastic Weight Consolidation on standard benchmarks. These desirable theoretical and empirical properties position GLNs as a complementary technique to contemporary offline deep learning methods.