# Improving the Accuracy and Robustness of CNNs Using a Deep CCA Neural Data Regularizer

arXiv: https://arxiv.org/abs/2209.02582 · HTML: https://mlsanity.com/paper/2209.02582/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 24 references that are in the map.
- First posted on arXiv 6 Sep 2022.

## Builds on (most cited first, 8 of 8)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [Adversarial examples in the physical world](https://mlsanity.com/paper/1607.02533/) (2016)
- [ResNet strikes back: An improved training procedure in timm](https://mlsanity.com/paper/2110.00476/) (2021)
- [Towards robust vision by multi-task learning on monkey visual cortex](https://mlsanity.com/paper/2107.14344/) (2021)
- [On 1/n neural representation and robustness](https://mlsanity.com/paper/2012.04729/) (2020)
- [pylustrator: Code generation for reproducible figures for publication](https://mlsanity.com/paper/1910.00279/) (2019)

## Built on by (most cited first, 3 of 3)

- [Achieving More Human Brain-Like Vision via Human EEG Representational Alignment](https://mlsanity.com/paper/2401.17231/) (2024)
- [Teaching CORnet Human fMRI Representations for Enhanced Model-Brain Alignment](https://mlsanity.com/paper/2407.10414/) (2024)
- [Limited but consistent gains in adversarial robustness by co-training object recognition models with human EEG](https://mlsanity.com/paper/2409.03646/) (2024)

## Abstract (arXiv)

As convolutional neural networks (CNNs) become more accurate at object recognition, their representations become more similar to the primate visual system. This finding has inspired us and other researchers to ask if the implication also runs the other way: If CNN representations become more brain-like, does the network become more accurate? Previous attempts to address this question showed very modest gains in accuracy, owing in part to limitations of the regularization method. To overcome these limitations, we developed a new neural data regularizer for CNNs that uses Deep Canonical Correlation Analysis (DCCA) to optimize the resemblance of the CNN's image representations to that of the monkey visual cortex. Using this new neural data regularizer, we see much larger performance gains in both classification accuracy and within-super-class accuracy, as compared to the previous state-of-the-art neural data regularizers. These networks are also more robust to adversarial attacks than their unregularized counterparts. Together, these results confirm that neural data regularization can push CNN performance higher, and introduces a new method that obtains a larger performance boost.