# On 1/n neural representation and robustness

arXiv: https://arxiv.org/abs/2012.04729 · HTML: https://mlsanity.com/paper/2012.04729/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Dec 2020.

## Built on by (most cited first, 17 of 17)

- [The Devil is in the Frequency: Geminated Gestalt Autoencoder for Self-Supervised Visual Pre-Training](https://mlsanity.com/paper/2204.08227/) (2022)
- [Learning Efficient Coding of Natural Images with Maximum Manifold Capacity Representations](https://mlsanity.com/paper/2303.03307/) (2023)
- [Temperature Balancing, Layer-wise Weight Analysis, and Neural Network Training](https://mlsanity.com/paper/2312.00359/) (2023)
- [Investigating Power laws in Deep Representation Learning](https://mlsanity.com/paper/2202.05808/) (2022)
- [A Spectral Theory of Neural Prediction and Alignment](https://mlsanity.com/paper/2309.12821/) (2023)
- [Model Balancing Helps Low-data Training and Fine-tuning](https://mlsanity.com/paper/2410.12178/) (2024)
- [Improving the Accuracy and Robustness of CNNs Using a Deep CCA Neural Data Regularizer](https://mlsanity.com/paper/2209.02582/) (2022)
- [Optimal input representation in neural systems at the edge of chaos](https://mlsanity.com/paper/2107.05709/) (2021)
- [Different Spectral Representations in Optimized Artificial Neural Networks and Brains](https://mlsanity.com/paper/2208.10576/) (2022)
- [Isometric Representations in Neural Networks Improve Robustness](https://mlsanity.com/paper/2211.01236/) (2022)
- [Discretization of continuous input spaces in the hippocampal autoencoder](https://mlsanity.com/paper/2405.14600/) (2024)
- [Limited but consistent gains in adversarial robustness by co-training object recognition models with human EEG](https://mlsanity.com/paper/2409.03646/) (2024)
- [Exploring Biologically Inspired Mechanisms of Adversarial Robustness](https://mlsanity.com/paper/2405.00679/) (2024)
- [Sensory robustness through top-down feedback and neural stochasticity in recurrent vision models](https://mlsanity.com/paper/2508.07115/) (2025)
- [Retina gap junctions support the robust perception by warping neural representational geometries along the visual hierarchy](https://mlsanity.com/paper/2604.14200/) (2026)
- [Learning reshapes power-law anisotropy in internal representations](https://mlsanity.com/paper/2608.15239/) (2026)
- [Dynamics to decision: A mathematical theory of Lyapunov spectra and decision boundaries in deep classifiers](https://mlsanity.com/paper/2609.39190/) (2026)

## Abstract (arXiv)

Understanding the nature of representation in neural networks is a goal shared by neuroscience and machine learning. It is therefore exciting that both fields converge not only on shared questions but also on similar approaches. A pressing question in these areas is understanding how the structure of the representation used by neural networks affects both their generalization, and robustness to perturbations. In this work, we investigate the latter by juxtaposing experimental results regarding the covariance spectrum of neural representations in the mouse V1 (Stringer et al) with artificial neural networks. We use adversarial robustness to probe Stringer et al's theory regarding the causal role of a 1/n covariance spectrum. We empirically investigate the benefits such a neural code confers in neural networks, and illuminate its role in multi-layer architectures. Our results show that imposing the experimentally observed structure on artificial neural networks makes them more robust to adversarial attacks. Moreover, our findings complement the existing theory relating wide neural networks to kernel methods, by showing the role of intermediate representations.