# Guiding Sparse Neural Networks with Neurobiological Principles to Elicit Biologically Plausible Representations

arXiv: https://arxiv.org/abs/2603.03234 · HTML: https://mlsanity.com/paper/2603.03234/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 76 references that are in the map.
- First posted on arXiv 3 Mar 2026.

## Builds on (most cited first, 24 of 24)

- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [The Forward-Forward Algorithm: Some Preliminary Investigations](https://mlsanity.com/paper/2212.13345/) (2022)
- [Equilibrium Propagation: Bridging the Gap Between Energy-Based Models and Backpropagation](https://mlsanity.com/paper/1602.05179/) (2016)
- [Difference Target Propagation](https://mlsanity.com/paper/1412.7525/) (2014)
- [Assessing the Scalability of Biologically-Motivated Deep Learning Algorithms and Architectures](https://mlsanity.com/paper/1807.04587/) (2018)
- [To Compress or Not to Compress- Self-Supervised Learning and Information Theory: A Review](https://mlsanity.com/paper/2304.09355/) (2023)
- [Unsupervised Learning by Competing Hidden Units](https://mlsanity.com/paper/1806.10181/) (2018)
- [Detecting and Diagnosing Adversarial Images with Class-Conditional Capsule Reconstructions](https://mlsanity.com/paper/1907.02957/) (2019)
- [Analyzing the Performance of Multilayer Neural Networks for Object Recognition](https://mlsanity.com/paper/1407.1610/) (2014)
- [Biologically inspired protection of deep networks from adversarial attacks](https://mlsanity.com/paper/1703.09202/) (2017)
- [The Backpropagation Algorithm Implemented on Spiking Neuromorphic Hardware](https://mlsanity.com/paper/2106.07030/) (2021)
- [Kickback cuts Backprop's red-tape: Biologically plausible credit assignment in neural networks](https://mlsanity.com/paper/1411.6191/) (2014)
- [Nonlinear Hebbian learning as a unifying principle in receptive field formation](https://mlsanity.com/paper/1601.00701/) (2016)
- [Contrastive Hebbian Learning with Random Feedback Weights](https://mlsanity.com/paper/1806.07406/) (2018)
- [RAILS: A Robust Adversarial Immune-inspired Learning System](https://mlsanity.com/paper/2107.02840/) (2021)
- [Activation Learning by Local Competitions](https://mlsanity.com/paper/2209.13400/) (2022)
- [Unsupervised representation learning with Hebbian synaptic and structural plasticity in brain-like feedforward neural networks](https://mlsanity.com/paper/2406.04733/) (2024)
- [Learning Sparse, Distributed Representations using the Hebbian Principle](https://mlsanity.com/paper/1611.04228/) (2016)
- [Dendritic error backpropagation in deep cortical microcircuits](https://mlsanity.com/paper/1801.00062/) (2017)
- [A theory of learning with constrained weight-distribution](https://mlsanity.com/paper/2206.08933/) (2022)
- [Tensor decomposition of higher-order correlations by nonlinear Hebbian plasticity](https://mlsanity.com/paper/2106.15685/) (2021)
- [A theoretical framework for learning through structural plasticity](https://mlsanity.com/paper/2307.11735/) (2023)
- [Energy-Efficient Information Representation in MNIST Classification Using Biologically Inspired Learning](https://mlsanity.com/paper/2603.00588/) (2026)

## Built on by (most cited first, 1 of 1)

- [Constrained Hebbian Learning Supports Efficient Representational Allocation under Structural Constraints](https://mlsanity.com/paper/2607.16027/) (2026)

## Abstract (arXiv)

While deep neural networks (DNNs) have achieved remarkable performance in tasks such as image recognition, they often struggle with generalization, learning from few examples, and continuous adaptation - abilities inherent in biological neural systems. These challenges arise due to DNNs' failure to emulate the efficient, adaptive learning mechanisms of biological networks. To address these issues, we explore the integration of neurobiologically inspired assumptions in neural network learning. This study introduces a biologically inspired learning rule that naturally integrates neurobiological principles, including sparsity, lognormal weight distributions, and adherence to Dale's law, without requiring explicit enforcement. By aligning with these core neurobiological principles, our model enhances robustness against adversarial attacks and demonstrates superior generalization, particularly in few-shot learning scenarios. Notably, integrating these constraints leads to the emergence of biologically plausible neural representations, underscoring the efficacy of incorporating neurobiological assumptions into neural network design. Preliminary results suggest that this approach could extend from feature-specific to task-specific encoding, potentially offering insights into neural resource allocation for complex tasks.