# Learning From Brains How to Regularize Machines

arXiv: https://arxiv.org/abs/1911.05072 · HTML: https://mlsanity.com/paper/1911.05072/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Nov 2019.

## Built on by (most cited first, 30 of 38)

- [A Systematic Review of Robustness in Deep Learning for Computer Vision: Mind the gap?](https://mlsanity.com/paper/2112.00639/) (2021)
- [The Algonauts Project 2023 Challenge: How the Human Brain Makes Sense of Natural Scenes](https://mlsanity.com/paper/2301.03198/) (2023)
- [Towards robust vision by multi-task learning on monkey visual cortex](https://mlsanity.com/paper/2107.14344/) (2021)
- [From superposition to sparse codes: interpretable representations in neural networks](https://mlsanity.com/paper/2503.01824/) (2025)
- [Toward Learning Human-aligned Cross-domain Robust Models by Countering Misaligned Features](https://mlsanity.com/paper/2111.03740/) (2021)
- [The Sensorium competition on predicting large-scale mouse primary visual cortex activity](https://mlsanity.com/paper/2206.08666/) (2022)
- [V1T: large-scale mouse V1 response prediction using a Vision Transformer](https://mlsanity.com/paper/2302.03023/) (2023)
- [Combining Different V1 Brain Model Variants to Improve Robustness to Image Corruptions in CNNs](https://mlsanity.com/paper/2110.10645/) (2021)
- [Explaining V1 Properties with a Biologically Constrained Deep Learning Architecture](https://mlsanity.com/paper/2305.11275/) (2023)
- [Achieving More Human Brain-Like Vision via Human EEG Representational Alignment](https://mlsanity.com/paper/2401.17231/) (2024)
- [Beyond representational alignment with brain-guided language models for robust reasoning](https://mlsanity.com/paper/2606.11893/) (2026)
- [Empirical Advocacy of Bio-inspired Models for Robust Image Recognition](https://mlsanity.com/paper/2205.09037/) (2022)
- [Neuro-Inspired Deep Neural Networks with Sparse, Strong Activations](https://mlsanity.com/paper/2202.13074/) (2022)
- [IB-U-Nets: Improving medical image segmentation tasks with 3D Inductive Biased kernels](https://mlsanity.com/paper/2210.15949/) (2022)
- [Teaching CORnet Human fMRI Representations for Enhanced Model-Brain Alignment](https://mlsanity.com/paper/2407.10414/) (2024)
- [Alignment and Adversarial Robustness: Are More Human-Like Models More Secure?](https://mlsanity.com/paper/2502.12377/) (2025)
- [Bio-inspired Robustness: A Review](https://mlsanity.com/paper/2103.09265/) (2021)
- [Partial Graph Reasoning for Neural Network Regularization](https://mlsanity.com/paper/2106.01805/) (2021)
- [Improving Neural Predictivity in the Visual Cortex with Gated Recurrent Connections](https://mlsanity.com/paper/2203.11910/) (2022)
- [Mitigating Urban-Rural Disparities in Contrastive Representation Learning with Satellite Imagery](https://mlsanity.com/paper/2211.08672/) (2022)
- [Feature representations useful for predicting image memorability](https://mlsanity.com/paper/2303.07679/) (2023)
- [Exploiting Large Neuroimaging Datasets to Create Connectome-Constrained Approaches for more Robust, Efficient, and Adaptable Artificial Intelligence](https://mlsanity.com/paper/2305.17300/) (2023)
- [Probing Human Visual Robustness with Neurally-Guided Deep Neural Networks](https://mlsanity.com/paper/2405.02564/) (2024)
- [Limited but consistent gains in adversarial robustness by co-training object recognition models with human EEG](https://mlsanity.com/paper/2409.03646/) (2024)
- [Mice to Machines: Neural Representations from Visual Cortex for Domain Generalization](https://mlsanity.com/paper/2505.06886/) (2025)
- [Rethinking Brain Decoding with CLIP: The Role of Adversarial Robustness](https://mlsanity.com/paper/2607.03165/) (2026)
- [3D-OOCS: Learning Prostate Segmentation with Inductive Bias](https://mlsanity.com/paper/2110.15664/) (2021)
- [Self-Attention-Based Contextual Modulation Improves Neural System Identification](https://mlsanity.com/paper/2406.07843/) (2024)
- [Pixel-Based Similarities as an Alternative to Neural Data for Improving Convolutional Neural Network Adversarial Robustness](https://mlsanity.com/paper/2410.03952/) (2024)
- [NeuRN: Neuro-inspired Domain Generalization for Image Classification](https://mlsanity.com/paper/2505.06881/) (2025)

## Abstract (arXiv)

Despite impressive performance on numerous visual tasks, Convolutional Neural Networks (CNNs) --- unlike brains --- are often highly sensitive to small perturbations of their input, e.g. adversarial noise leading to erroneous decisions. We propose to regularize CNNs using large-scale neuroscience data to learn more robust neural features in terms of representational similarity. We presented natural images to mice and measured the responses of thousands of neurons from cortical visual areas. Next, we denoised the notoriously variable neural activity using strong predictive models trained on this large corpus of responses from the mouse visual system, and calculated the representational similarity for millions of pairs of images from the model's predictions. We then used the neural representation similarity to regularize CNNs trained on image classification by penalizing intermediate representations that deviated from neural ones. This preserved performance of baseline models when classifying images under standard benchmarks, while maintaining substantially higher performance compared to baseline or control models when classifying noisy images. Moreover, the models regularized with cortical representations also improved model robustness in terms of adversarial attacks. This demonstrates that regularizing with neural data can be an effective tool to create an inductive bias towards more robust inference.