# Contributions of Shape, Texture, and Color in Visual Recognition

arXiv: https://arxiv.org/abs/2207.09510 · HTML: https://mlsanity.com/paper/2207.09510/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 62 references that are in the map.
- First posted on arXiv 19 Jul 2022.

## Builds on (most cited first, 30 of 31)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [Matching Networks for One Shot Learning](https://mlsanity.com/paper/1606.04080/) (2016)
- [Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization](https://mlsanity.com/paper/1703.06868/) (2017)
- [Vision Transformers for Dense Prediction](https://mlsanity.com/paper/2103.13413/) (2021)
- [Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfer](https://mlsanity.com/paper/1907.01341/) (2019)
- [ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness](https://mlsanity.com/paper/1811.12231/) (2018)
- [Learning to Compare: Relation Network for Few-Shot Learning](https://mlsanity.com/paper/1711.06025/) (2017)
- [ConceptNet 5.5: An Open Multilingual Graph of General Knowledge](https://mlsanity.com/paper/1612.03975/) (2016)
- [Learning to learn by gradient descent by gradient descent](https://mlsanity.com/paper/1606.04474/) (2016)
- [Towards Open World Object Detection](https://mlsanity.com/paper/2103.02603/) (2021)
- [Approximating CNNs with Bag-of-local-Features models works surprisingly well on ImageNet](https://mlsanity.com/paper/1904.00760/) (2019)
- [Towards Open World Recognition](https://mlsanity.com/paper/1412.5687/) (2014)
- [Visual Object Networks: Image Generation with Disentangled 3D Representation](https://mlsanity.com/paper/1812.02725/) (2018)
- [Shape-Texture Debiased Neural Network Training](https://mlsanity.com/paper/2010.05981/) (2020)
- [Unsupervised Meta-Learning For Few-Shot Image Classification](https://mlsanity.com/paper/1811.11819/) (2018)
- [Open-World Entity Segmentation](https://mlsanity.com/paper/2107.14228/) (2021)
- [Deep Neural Networks Rival the Representation of Primate IT Cortex for Core Visual Object Recognition](https://mlsanity.com/paper/1406.3284/) (2014)
- [Learning Compositional Representations for Few-Shot Recognition](https://mlsanity.com/paper/1812.09213/) (2018)
- [A Peek Into the Reasoning of Neural Networks: Interpreting with Structural Visual Concepts](https://mlsanity.com/paper/2105.00290/) (2021)
- [A Unified approach for Conventional Zero-shot, Generalized Zero-shot and Few-shot Learning](https://mlsanity.com/paper/1706.08653/) (2017)
- [Recent Advances in Zero-shot Recognition](https://mlsanity.com/paper/1710.04837/) (2017)
- [Zero-shot Synthesis with Group-Supervised Learning](https://mlsanity.com/paper/2009.06586/) (2020)
- [Disentangling 3D Prototypical Networks For Few-Shot Concept Learning](https://mlsanity.com/paper/2011.03367/) (2020)
- [Disentangled Feature Representation for Few-shot Image Classification](https://mlsanity.com/paper/2109.12548/) (2021)

## Built on by (most cited first, 16 of 16)

- [Texture Learning Domain Randomization for Domain Generalized Segmentation](https://mlsanity.com/paper/2303.11546/) (2023)
- [ImageNet-trained CNNs are not biased towards texture: Revisiting feature reliance through controlled suppression](https://mlsanity.com/paper/2509.20234/) (2025)
- [What You See Is What Matters: A Novel Visual and Physics-Based Metric for Evaluating Video Generation Quality](https://mlsanity.com/paper/2411.13609/) (2024)
- [Intriguing Properties of Diffusion Models: An Empirical Study of the Natural Attack Capability in Text-to-Image Generative Models](https://mlsanity.com/paper/2308.15692/) (2023)
- [Data Generation Scheme for Thermal Modality with Edge-Guided Adversarial Conditional Diffusion Model](https://mlsanity.com/paper/2408.03748/) (2024)
- [MAGIC: Mastering Physical Adversarial Generation in Context through Collaborative LLM Agents](https://mlsanity.com/paper/2412.08014/) (2024)
- [Discovering Semantic Subdimensions through Disentangled Conceptual Representations](https://mlsanity.com/paper/2508.21436/) (2025)
- [On the Reliability of Cue Conflict and Beyond](https://mlsanity.com/paper/2603.10834/) (2026)
- [The Influences of Color and Shape Features in Visual Contrastive Learning](https://mlsanity.com/paper/2301.12459/) (2023)
- [Depth Insight -- Contribution of Different Features to Indoor Single-image Depth Estimation](https://mlsanity.com/paper/2311.10042/) (2023)
- [On the Relationship Between Double Descent of CNNs and Shape/Texture Bias Under Learning Process](https://mlsanity.com/paper/2503.02302/) (2025)
- [Haphazard Inputs as Images in Online Learning](https://mlsanity.com/paper/2504.02912/) (2025)
- [Humanoid-inspired Causal Representation Learning for Domain Generalization](https://mlsanity.com/paper/2510.16382/) (2025)
- [T2I-Based Physical-World Appearance Attack against Traffic Sign Recognition Systems in Autonomous Driving](https://mlsanity.com/paper/2511.12956/) (2025)
- [Visual Categorization Across Minds and Models: Cognitive Analysis of Human Labeling and Neuro-Symbolic Integration](https://mlsanity.com/paper/2512.09340/) (2025)
- [Anatomy of a failure: When, how, and why deep vision fails in scientific domains](https://mlsanity.com/paper/2605.04231/) (2026)

## Abstract (arXiv)

We investigate the contributions of three important features of the human visual system (HVS)~ -- ~shape, texture, and color ~ -- ~to object classification. We build a humanoid vision engine (HVE) that explicitly and separately computes shape, texture, and color features from images. The resulting feature vectors are then concatenated to support the final classification. We show that HVE can summarize and rank-order the contributions of the three features to object recognition. We use human experiments to confirm that both HVE and humans predominantly use some specific features to support the classification of specific classes (e.g., texture is the dominant feature to distinguish a zebra from other quadrupeds, both for humans and HVE). With the help of HVE, given any environment (dataset), we can summarize the most important features for the whole task (task-specific; e.g., color is the most important feature overall for classification with the CUB dataset), and for each class (class-specific; e.g., shape is the most important feature to recognize boats in the iLab-20M dataset). To demonstrate more usefulness of HVE, we use it to simulate the open-world zero-shot learning ability of humans with no attribute labeling. Finally, we show that HVE can also simulate human imagination ability with the combination of different features. We will open-source the HVE engine and corresponding datasets.