mlsanityData to 14 Oct 2026

Spatial-frequency channels, shape bias, and adversarial robustness

arXiv 2309.13190 ↗First posted 22 Sep 2023cs.LG cs.CV eess.IVcited by 28

Not on the map. Why?

Abstract (arXiv)

What spatial frequency information do humans and neural networks use to recognize objects? In neuroscience, critical band masking is an established tool that can reveal the frequency-selective filters used for object recognition. Critical band masking measures the sensitivity of recognition performance to noise added at each spatial frequency. Existing critical band masking studies show that humans recognize periodic patterns (gratings) and letters by means of a spatial-frequency filter (or "channel") that has a frequency bandwidth of one octave (doubling of frequency). Here, we introduce critical band masking as a task for network-human comparison and test 14 humans and 76 neural networks on 16-way ImageNet categorization in the presence of narrowband noise. We find that humans recognize objects in natural images using the same one-octave-wide channel that they use for letters and gratings, making it a canonical feature of human object recognition. Unlike humans, the neural network channel is very broad, 2-4 times wider than the human channel. Thus, noise at certain high and low frequencies will impair network performance and spare human performance. Adversarial and augmented-image training are commonly used to increase network robustness and shape bias. Does this training align network and human object recognition channels? Three network channel properties (bandwidth, center frequency, peak noise sensitivity) correlate strongly with shape bias (51% variance explained) and robustness of adversarially-trained networks (66% variance explained). Adversarial training increases robustness but expands the channel bandwidth even further beyond the human bandwidth. Thus, critical band masking reveals that the network channel is more than twice as wide as the human channel, and that adversarial training only makes it worse. Networks with narrower channels might be more robust.

Key facts. Cited by 28 papers in the map (data through 14 Oct 2026). Builds on 29 of its 61 references that are in the map. First posted on arXiv 22 Sep 2023.

Builds on

29 of 61 references.

References · 29

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  2. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  3. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  4. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  5. 2013Intriguing properties of neural networkscited by 5,155
  6. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,287
  7. 2020Shortcut Learning in Deep Neural Networkscited by 1,969
  8. 2018Unsupervised Feature Learning via Non-Parametric Instance-level Discriminationcited by 1,845
  9. 2018ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustnesscited by 1,432
  10. 2018On the Spectral Bias of Neural Networkscited by 1,341
  11. 2019Self-training with Noisy Student improves ImageNet classificationcited by 1,311
  12. 2020What Makes for Good Views for Contrastive Learning?cited by 846
17 more
  1. 2019Self-Supervised Learning of Pretext-Invariant Representationscited by 808
  2. 2014Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Imagescited by 776
  3. 2019Big Transfer (BiT): General Visual Representation Learningcited by 769
  4. 2023Scaling Vision Transformers to 22 Billion Parameterscited by 694
  5. 2020Do Adversarially Robust ImageNet Models Transfer Better?cited by 316
  6. 2019High Frequency Component Helps Explain the Generalization of Convolutional Neural Networkscited by 310
  7. 2019A Fourier Perspective on Model Robustness in Computer Visioncited by 288
  8. 2018Generalisation in humans and deep neural networkscited by 237
  9. 2019Billion-scale semi-supervised learning for image classificationcited by 232
  10. 2018Adversarial Robustness Toolbox v1.0.0cited by 215
  11. 2021Partial success in closing the gap between human and machine visioncited by 185
  12. 2021Are Convolutional Neural Networks or Transformers more like human vision?cited by 109
  13. 2018Adversarial Examples that Fool both Computer Vision and Time-Limited Humanscited by 73
  14. 2018Humans can decipher adversarial imagescited by 31
  15. 2021A Frequency Perspective of Adversarial Robustnesscited by 28
  16. 2023An Extended Study of Human-like Behavior under Adversarial Trainingcited by 9
  17. 2021Impact of Spatial Frequency Based Constraints on Adversarial Robustnesscited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 3; 2024: 12; 2025: 8; 2026: 5.

28 papers.

Sort:

By area:

  1. 2023On the Foundations of Shortcut Learningcited by 65
  2. 2023Intriguing properties of generative classifierscited by 52
  3. 2024Adversarial Robustness Limits via Scaling-Law and Human-Alignment Studiescited by 38
  4. 2024Frequency-Adaptive Dilated Convolution for Semantic Segmentationcited by 34
  5. 2024Can We Talk Models Into Seeing the World Differently?cited by 23
  6. 2024Can Biases in ImageNet Models Explain Generalization?cited by 20
  7. 2025Frequency Dynamic Convolution for Dense Image Predictioncited by 19
  8. 2025ImageNet-trained CNNs are not biased towards texture: Revisiting feature reliance through controlled suppressioncited by 13
  9. 2024Discriminating image representations with principal distortionscited by 8
  10. 2024Towards Combating Frequency Simplicity-biased Learning for Domain Generalizationcited by 6
  11. 2024Towards Adversarial Robustness via Debiased High-Confidence Logit Alignmentcited by 5
  12. 2024Quantifying the Limits of Segmentation Foundation Models: Modeling Challenges in Segmenting Tree-Like and Low-Contrast Objectscited by 5
16 more
  1. 2025Shifting the Paradigm: A Diffeomorphism Between Time Series Data Manifolds for Achieving Shift-Invariancy in Deep Learningcited by 5
  2. 2024AdaNCA: Neural Cellular Automata As Adaptors For More Robust Vision Transformercited by 4
  3. 2025Visual Anagrams Reveal Hidden Differences in Holistic Shape Processing Across Vision Modelscited by 3
  4. 2024DT-JRD: Deep Transformer based Just Recognizable Difference Prediction Model for Video Coding for Machinescited by 2
  5. 2025Alignment and Adversarial Robustness: Are More Human-Like Models More Secure?cited by 2
  6. 2023Layerwise complexity-matched learning yields an improved model of cortical area V2cited by 1
  7. 2024DCT-CryptoNets: Scaling Private Inference in the Frequency Domaincited by 1
  8. 2024VFA: Vision Frequency Analysis of Foundation Models and Humancited by 1
  9. 2025Contrast Sensitivity in Multimodal Large Language Models: A Psychophysics-Inspired Evaluationcited by 1
  10. 2026Low-Pass Filtering Improves Behavioral Alignment of Vision Modelscited by 1
  11. 2026Rethinking Brain Decoding with CLIP: The Role of Adversarial Robustnesscited by 1
  12. 2025Spectral-Adaptive Modulation Networks for Visual Perceptioncited by 0
  13. 2025Quantifying task-relevant representational similarity using decision variable correlationcited by 0
  14. 2026Anatomy of a failure: When, how, and why deep vision fails in scientific domainscited by 0
  15. 2026Low-Frequency Shortcuts in Texture-Driven Visual Learningcited by 0
  16. 2026Adaptive Causal Alignment for High-Confidence Adversarial Trainingcited by 0