# Spatial-frequency channels, shape bias, and adversarial robustness

arXiv: https://arxiv.org/abs/2309.13190 · HTML: https://mlsanity.com/paper/2309.13190/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 61 references that are in the map.
- First posted on arXiv 22 Sep 2023.

## Builds on (most cited first, 29 of 29)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Improved Baselines with Momentum Contrastive Learning](https://mlsanity.com/paper/2003.04297/) (2020)
- [Shortcut Learning in Deep Neural Networks](https://mlsanity.com/paper/2004.07780/) (2020)
- [Unsupervised Feature Learning via Non-Parametric Instance-level Discrimination](https://mlsanity.com/paper/1805.01978/) (2018)
- [ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness](https://mlsanity.com/paper/1811.12231/) (2018)
- [On the Spectral Bias of Neural Networks](https://mlsanity.com/paper/1806.08734/) (2018)
- [Self-training with Noisy Student improves ImageNet classification](https://mlsanity.com/paper/1911.04252/) (2019)
- [What Makes for Good Views for Contrastive Learning?](https://mlsanity.com/paper/2005.10243/) (2020)
- [Self-Supervised Learning of Pretext-Invariant Representations](https://mlsanity.com/paper/1912.01991/) (2019)
- [Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Images](https://mlsanity.com/paper/1412.1897/) (2014)
- [Big Transfer (BiT): General Visual Representation Learning](https://mlsanity.com/paper/1912.11370/) (2019)
- [Scaling Vision Transformers to 22 Billion Parameters](https://mlsanity.com/paper/2302.05442/) (2023)
- [Do Adversarially Robust ImageNet Models Transfer Better?](https://mlsanity.com/paper/2007.08489/) (2020)
- [High Frequency Component Helps Explain the Generalization of Convolutional Neural Networks](https://mlsanity.com/paper/1905.13545/) (2019)
- [A Fourier Perspective on Model Robustness in Computer Vision](https://mlsanity.com/paper/1906.08988/) (2019)
- [Generalisation in humans and deep neural networks](https://mlsanity.com/paper/1808.08750/) (2018)
- [Billion-scale semi-supervised learning for image classification](https://mlsanity.com/paper/1905.00546/) (2019)
- [Adversarial Robustness Toolbox v1.0.0](https://mlsanity.com/paper/1807.01069/) (2018)
- [Partial success in closing the gap between human and machine vision](https://mlsanity.com/paper/2106.07411/) (2021)
- [Are Convolutional Neural Networks or Transformers more like human vision?](https://mlsanity.com/paper/2105.07197/) (2021)
- [Adversarial Examples that Fool both Computer Vision and Time-Limited Humans](https://mlsanity.com/paper/1802.08195/) (2018)
- [Humans can decipher adversarial images](https://mlsanity.com/paper/1809.04120/) (2018)
- [A Frequency Perspective of Adversarial Robustness](https://mlsanity.com/paper/2111.00861/) (2021)
- [An Extended Study of Human-like Behavior under Adversarial Training](https://mlsanity.com/paper/2303.12669/) (2023)
- [Impact of Spatial Frequency Based Constraints on Adversarial Robustness](https://mlsanity.com/paper/2104.12679/) (2021)

## Built on by (most cited first, 28 of 28)

- [On the Foundations of Shortcut Learning](https://mlsanity.com/paper/2310.16228/) (2023)
- [Intriguing properties of generative classifiers](https://mlsanity.com/paper/2309.16779/) (2023)
- [Adversarial Robustness Limits via Scaling-Law and Human-Alignment Studies](https://mlsanity.com/paper/2404.09349/) (2024)
- [Frequency-Adaptive Dilated Convolution for Semantic Segmentation](https://mlsanity.com/paper/2403.05369/) (2024)
- [Can We Talk Models Into Seeing the World Differently?](https://mlsanity.com/paper/2403.09193/) (2024)
- [Can Biases in ImageNet Models Explain Generalization?](https://mlsanity.com/paper/2404.01509/) (2024)
- [Frequency Dynamic Convolution for Dense Image Prediction](https://mlsanity.com/paper/2503.18783/) (2025)
- [ImageNet-trained CNNs are not biased towards texture: Revisiting feature reliance through controlled suppression](https://mlsanity.com/paper/2509.20234/) (2025)
- [Discriminating image representations with principal distortions](https://mlsanity.com/paper/2410.15433/) (2024)
- [Towards Combating Frequency Simplicity-biased Learning for Domain Generalization](https://mlsanity.com/paper/2410.16146/) (2024)
- [Towards Adversarial Robustness via Debiased High-Confidence Logit Alignment](https://mlsanity.com/paper/2408.06079/) (2024)
- [Quantifying the Limits of Segmentation Foundation Models: Modeling Challenges in Segmenting Tree-Like and Low-Contrast Objects](https://mlsanity.com/paper/2412.04243/) (2024)
- [Shifting the Paradigm: A Diffeomorphism Between Time Series Data Manifolds for Achieving Shift-Invariancy in Deep Learning](https://mlsanity.com/paper/2502.19921/) (2025)
- [AdaNCA: Neural Cellular Automata As Adaptors For More Robust Vision Transformer](https://mlsanity.com/paper/2406.08298/) (2024)
- [Visual Anagrams Reveal Hidden Differences in Holistic Shape Processing Across Vision Models](https://mlsanity.com/paper/2507.00493/) (2025)
- [DT-JRD: Deep Transformer based Just Recognizable Difference Prediction Model for Video Coding for Machines](https://mlsanity.com/paper/2411.09308/) (2024)
- [Alignment and Adversarial Robustness: Are More Human-Like Models More Secure?](https://mlsanity.com/paper/2502.12377/) (2025)
- [Layerwise complexity-matched learning yields an improved model of cortical area V2](https://mlsanity.com/paper/2312.11436/) (2023)
- [DCT-CryptoNets: Scaling Private Inference in the Frequency Domain](https://mlsanity.com/paper/2408.15231/) (2024)
- [VFA: Vision Frequency Analysis of Foundation Models and Human](https://mlsanity.com/paper/2409.05817/) (2024)
- [Contrast Sensitivity in Multimodal Large Language Models: A Psychophysics-Inspired Evaluation](https://mlsanity.com/paper/2508.10367/) (2025)
- [Low-Pass Filtering Improves Behavioral Alignment of Vision Models](https://mlsanity.com/paper/2602.13859/) (2026)
- [Rethinking Brain Decoding with CLIP: The Role of Adversarial Robustness](https://mlsanity.com/paper/2607.03165/) (2026)
- [Spectral-Adaptive Modulation Networks for Visual Perception](https://mlsanity.com/paper/2503.23947/) (2025)
- [Quantifying task-relevant representational similarity using decision variable correlation](https://mlsanity.com/paper/2506.02164/) (2025)
- [Anatomy of a failure: When, how, and why deep vision fails in scientific domains](https://mlsanity.com/paper/2605.04231/) (2026)
- [Low-Frequency Shortcuts in Texture-Driven Visual Learning](https://mlsanity.com/paper/2606.03493/) (2026)
- [Adaptive Causal Alignment for High-Confidence Adversarial Training](https://mlsanity.com/paper/2606.03925/) (2026)

## Abstract (arXiv)

What spatial frequency information do humans and neural networks use to recognize objects? In neuroscience, critical band masking is an established tool that can reveal the frequency-selective filters used for object recognition. Critical band masking measures the sensitivity of recognition performance to noise added at each spatial frequency. Existing critical band masking studies show that humans recognize periodic patterns (gratings) and letters by means of a spatial-frequency filter (or "channel") that has a frequency bandwidth of one octave (doubling of frequency). Here, we introduce critical band masking as a task for network-human comparison and test 14 humans and 76 neural networks on 16-way ImageNet categorization in the presence of narrowband noise. We find that humans recognize objects in natural images using the same one-octave-wide channel that they use for letters and gratings, making it a canonical feature of human object recognition. Unlike humans, the neural network channel is very broad, 2-4 times wider than the human channel. Thus, noise at certain high and low frequencies will impair network performance and spare human performance. Adversarial and augmented-image training are commonly used to increase network robustness and shape bias. Does this training align network and human object recognition channels? Three network channel properties (bandwidth, center frequency, peak noise sensitivity) correlate strongly with shape bias (51% variance explained) and robustness of adversarially-trained networks (66% variance explained). Adversarial training increases robustness but expands the channel bandwidth even further beyond the human bandwidth. Thus, critical band masking reveals that the network channel is more than twice as wide as the human channel, and that adversarial training only makes it worse. Networks with narrower channels might be more robust.