# Deep Convolutional Neural Networks in the Face of Caricature: Identity and Image Revealed

arXiv: https://arxiv.org/abs/1812.10902 · HTML: https://mlsanity.com/paper/1812.10902/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Dec 2018.

## Built on by (most cited first, 16 of 16)

- [Neither hype nor gloom do DNNs justice](https://mlsanity.com/paper/2312.05355/) (2023)
- [PASS: Protected Attribute Suppression System for Mitigating Bias in Face Recognition](https://mlsanity.com/paper/2108.03764/) (2021)
- [Gradient Attention Balance Network: Mitigating Face Recognition Racial Bias via Gradient Attention](https://mlsanity.com/paper/2304.02284/) (2023)
- [Distill and De-bias: Mitigating Bias in Face Verification using Knowledge Distillation](https://mlsanity.com/paper/2112.09786/) (2021)
- [Twin identification over viewpoint change: A deep convolutional neural network surpasses humans](https://mlsanity.com/paper/2207.05316/) (2022)
- [Dissecting Human Body Representations in Deep Networks Trained for Person Identification](https://mlsanity.com/paper/2502.15934/) (2025)
- [Attributes Shape the Embedding Space of Face Recognition Models](https://mlsanity.com/paper/2507.11372/) (2025)
- [A Quantitative Evaluation of the Expressivity of BMI, Pose and Gender in Body Embeddings for Recognition and Identification](https://mlsanity.com/paper/2503.06451/) (2025)
- [Beyond the limitations of any imaginable mechanism: large language models and psycholinguistics](https://mlsanity.com/paper/2303.00077/) (2023)
- [Configural processing as an optimized strategy for robust object recognition in neural networks](https://mlsanity.com/paper/2407.19072/) (2024)
- [Fixing the problems of deep neural networks will require better training data and learning algorithms](https://mlsanity.com/paper/2311.12819/) (2023)
- [Revisiting Marr in Face: The Building of 2D--2.5D--3D Representations in Deep Neural Networks](https://mlsanity.com/paper/2411.16148/) (2024)
- [A solution to generalized learning from small training sets found in infants repeated visual experiences of individual objects](https://mlsanity.com/paper/2510.15060/) (2025)
- [AttriBE: Quantifying Attribute Expressivity in Body Embeddings for Recognition and Identification](https://mlsanity.com/paper/2604.27218/) (2026)
- [Perceived face similarity reflects inverse-generative and naturalistic discriminative objectives](https://mlsanity.com/paper/2605.12619/) (2026)
- [Beyond EER: Multi-Dimensional Evaluation of Information Leakage in Speaker De-Identification](https://mlsanity.com/paper/2609.18673/) (2026)

## Abstract (arXiv)

Real-world face recognition requires an ability to perceive the unique features of an individual face across multiple, variable images. The primate visual system solves the problem of image invariance using cascades of neurons that convert images of faces into categorical representations of facial identity. Deep convolutional neural networks (DCNNs) also create generalizable face representations, but with cascades of simulated neurons. DCNN representations can be examined in a multidimensional "face space", with identities and image parameters quantified via their projections onto the axes that define the space. We examined the organization of viewpoint, illumination, gender, and identity in this space. We show that the network creates a highly organized, hierarchically nested, face similarity structure in which information about face identity and imaging characteristics coexist. Natural image variation is accommodated in this hierarchy, with face identity nested under gender, illumination nested under identity, and viewpoint nested under illumination. To examine identity, we caricatured faces and found that network identification accuracy increased with caricature level, and--mimicking human perception--a caricatured distortion of a face "resembled" its veridical counterpart. Caricatures improved performance by moving the identity away from other identities in the face space and minimizing the effects of illumination and viewpoint. Deep networks produce face representations that solve long-standing computational problems in generalized face recognition. They also provide a unitary theoretical framework for reconciling decades of behavioral and neural results that emphasized either the image or the object/face in representations, without understanding how a neural code could seamlessly accommodate both.