# Winning the lottery with neural connectivity constraints: faster learning across cognitive tasks with spatially constrained sparse RNNs

arXiv: https://arxiv.org/abs/2207.03523 · HTML: https://mlsanity.com/paper/2207.03523/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Jul 2022.

## Built on by (most cited first, 10 of 10)

- [How connectivity structure shapes rich and lazy learning in neural circuits](https://mlsanity.com/paper/2310.08513/) (2023)
- [Self-Supervised Learning of Representations for Space Generates Multi-Modular Grid Cells](https://mlsanity.com/paper/2311.02316/) (2023)
- [Growing Brains: Co-emergence of Anatomical and Functional Modularity in Recurrent Neural Networks](https://mlsanity.com/paper/2310.07711/) (2023)
- [Dale meets Langevin: A Multiplicative Denoising Diffusion Model](https://mlsanity.com/paper/2510.02730/) (2025)
- [Modularity is the Bedrock of Natural and Artificial Intelligence](https://mlsanity.com/paper/2602.18960/) (2026)
- [Behavior quantification as the missing link between fields: Tools for digital psychiatry and their role in the future of neurobiology](https://mlsanity.com/paper/2305.15385/) (2023)
- [Measuring and Controlling Solution Degeneracy across Task-Trained Recurrent Neural Networks](https://mlsanity.com/paper/2410.03972/) (2024)
- [Paradoxical noise preference in RNNs](https://mlsanity.com/paper/2601.04539/) (2026)
- [Harnessing cortical geometry, wiring, and function as inductive biases for recurrent neural networks](https://mlsanity.com/paper/2606.14975/) (2026)
- [Effective pruning of task-trained recurrent neural networks using noisy fluctuations and connection rescaling](https://mlsanity.com/paper/2608.05464/) (2026)

## Abstract (arXiv)

Recurrent neural networks (RNNs) are often used to model circuits in the brain, and can solve a variety of difficult computational problems requiring memory, error-correction, or selection [Hopfield, 1982, Maass et al., 2002, Maass, 2011]. However, fully-connected RNNs contrast structurally with their biological counterparts, which are extremely sparse (~0.1%). Motivated by the neocortex, where neural connectivity is constrained by physical distance along cortical sheets and other synaptic wiring costs, we introduce locality masked RNNs (LM-RNNs) that utilize task-agnostic predetermined graphs with sparsity as low as 4%. We study LM-RNNs in a multitask learning setting relevant to cognitive systems neuroscience with a commonly used set of tasks, 20-Cog-tasks [Yang et al., 2019]. We show through reductio ad absurdum that 20-Cog-tasks can be solved by a small pool of separated autapses that we can mechanistically analyze and understand. Thus, these tasks fall short of the goal of inducing complex recurrent dynamics and modular structure in RNNs. We next contribute a new cognitive multi-task battery, Mod-Cog, consisting of upto 132 tasks that expands by 7-fold the number of tasks and task-complexity of 20-Cog-tasks. Importantly, while autapses can solve the simple 20-Cog-tasks, the expanded task-set requires richer neural architectures and continuous attractor dynamics. On these tasks, we show that LM-RNNs with an optimal sparsity result in faster training and better data-efficiency than fully connected networks.