# Perceptual misalignment of texture representations in convolutional neural networks

arXiv: https://arxiv.org/abs/2604.01341 · HTML: https://mlsanity.com/paper/2604.01341/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 83 references that are in the map.
- First posted on arXiv 1 Apr 2026.

## Builds on (most cited first, 24 of 24)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [Describing Textures in the Wild](https://mlsanity.com/paper/1311.3618/) (2013)
- [ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness](https://mlsanity.com/paper/1811.12231/) (2018)
- [Improved Texture Networks: Maximizing Quality and Diversity in Feed-forward Stylization and Texture Synthesis](https://mlsanity.com/paper/1701.02096/) (2017)
- [Partial success in closing the gap between human and machine vision](https://mlsanity.com/paper/2106.07411/) (2021)
- [Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Brain's Ventral Visual Pathway](https://mlsanity.com/paper/1411.6422/) (2014)
- [Diversified Texture Synthesis with Feed-forward Networks](https://mlsanity.com/paper/1703.01664/) (2017)
- [Shape or Texture: Understanding Discriminative Features in CNNs](https://mlsanity.com/paper/2101.11604/) (2021)
- [From BoW to CNN: Two Decades of Texture Representation for Texture Classification](https://mlsanity.com/paper/1801.10324/) (2018)
- [Texture Interpolation for Probing Visual Perception](https://mlsanity.com/paper/2006.03698/) (2020)
- [Texture Mixer: A Network for Controllable Synthesis and Interpolation of Texture](https://mlsanity.com/paper/1901.03447/) (2019)
- [Are Deep Neural Networks Adequate Behavioural Models of Human Visual Perception?](https://mlsanity.com/paper/2305.17023/) (2023)
- [Texture Synthesis Through Convolutional Neural Networks and Spectrum Constraints](https://mlsanity.com/paper/1605.01141/) (2016)
- [Incorporating long-range consistency in CNN-based texture generation](https://mlsanity.com/paper/1606.01286/) (2016)
- [Infinite Texture: Text-guided High Resolution Diffusion Texture Synthesis](https://mlsanity.com/paper/2405.08210/) (2024)
- [Paying U-Attention to Textures: Multi-Stage Hourglass Vision Transformer for Universal Texture Synthesis](https://mlsanity.com/paper/2202.11703/) (2022)
- [Prune and distill: similar reformatting of image information along rat visual cortex and deep neural networks](https://mlsanity.com/paper/2205.13816/) (2022)

## Built on by (most cited first, 2 of 2)

- [A Fourier perspective on the learning dynamics of neural networks: from sample complexities to mechanistic insights](https://mlsanity.com/paper/2605.16913/) (2026)
- [Texture Representations in Deep Vision Models: Comparing CNNs, Vision Transformers, and Human Perception](https://mlsanity.com/paper/2607.08321/) (2026)

## Abstract (arXiv)

Mathematical modeling of visual textures traces back to Julesz's intuition that texture perception in humans is based on local correlations between image features. An influential approach for texture analysis and generation generalizes this notion to linear correlations between the nonlinear features computed by convolutional neural networks (CNNs), compiled into Gram matrices. Given that CNNs are often used as models for the visual system, it is natural to ask whether such "texture representations" spontaneously align with the textures' perceptual content, and in particular whether those CNNs that are regarded as better models for the visual system also possess more human-like texture representations. Here we quantify the perceptual content captured by feature correlations computed for a diverse pool of CNNs, and we compare it to the models' perceptual alignment with the mammalian visual system as measured by Brain-Score. Surprisingly, we find that there is no connection between conventional measures of CNN quality as a model of the visual system and its alignment with human texture perception. We conclude that texture perception involves mechanisms that are distinct from those that are commonly modeled using approaches based on CNNs trained on object recognition, possibly depending on the integration of contextual information.