# Synthesizing Images on Perceptual Boundaries of ANNs for Uncovering Human Perceptual Variability on Facial Expressions

arXiv: https://arxiv.org/abs/2507.14549 · HTML: https://mlsanity.com/paper/2507.14549/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 21 of its 37 references that are in the map.
- First posted on arXiv 19 Jul 2025.

## Builds on (most cited first, 21 of 21)

- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [All are Worth Words: A ViT Backbone for Diffusion Models](https://mlsanity.com/paper/2209.12152/) (2022)
- [DreamSim: Learning New Dimensions of Human Visual Similarity using Synthetic Data](https://mlsanity.com/paper/2306.09344/) (2023)
- [FreeDoM: Training-Free Energy-Guided Conditional Diffusion Model](https://mlsanity.com/paper/2303.09833/) (2023)
- [Diffusion Models for Counterfactual Explanations](https://mlsanity.com/paper/2203.15636/) (2022)
- [Human alignment of neural network representations](https://mlsanity.com/paper/2211.01201/) (2022)
- [Guidance with Spherical Gaussian Constraint for Conditional Diffusion](https://mlsanity.com/paper/2402.03201/) (2024)
- [Adversarial Counterfactual Visual Explanations](https://mlsanity.com/paper/2303.09962/) (2023)
- [Dimensions underlying the representational alignment of deep neural networks with humans](https://mlsanity.com/paper/2406.19087/) (2024)
- [Elucidating The Design Space of Classifier-Guided Diffusion Generation](https://mlsanity.com/paper/2310.11311/) (2023)
- [When Does Perceptual Alignment Benefit Vision Representations?](https://mlsanity.com/paper/2410.10817/) (2024)
- [Revealing interpretable object representations from human behavior](https://mlsanity.com/paper/1901.02915/) (2019)
- [VICE: Variational Interpretable Concept Embeddings](https://mlsanity.com/paper/2205.00756/) (2022)
- [DreaMR: Diffusion-driven Counterfactual Explanation for Functional MRI](https://mlsanity.com/paper/2307.09547/) (2023)
- [Measuring Representational Robustness of Neural Networks Through Shared Invariances](https://mlsanity.com/paper/2206.11939/) (2022)
- [Testing the limits of natural language models for predicting human language judgments](https://mlsanity.com/paper/2204.03592/) (2022)
- [Diffusion-based Visual Counterfactual Explanations -- Towards Systematic Quantitative Evaluation](https://mlsanity.com/paper/2308.06100/) (2023)
- [CoCoG: Controllable Visual Stimuli Generation based on Human Concept Representations](https://mlsanity.com/paper/2404.16482/) (2024)
- [CoCoG-2: Controllable generation of visual stimuli for understanding human concept representation](https://mlsanity.com/paper/2407.14949/) (2024)
- [Do Invariances in Deep Neural Networks Align with Human Perception?](https://mlsanity.com/paper/2111.14726/) (2021)

## Abstract (arXiv)

A fundamental challenge in affective cognitive science is to develop models that accurately capture the relationship between external emotional stimuli and human internal experiences. While ANNs have demonstrated remarkable accuracy in facial expression recognition, their ability to model inter-individual differences in human perception remains underexplored. This study investigates the phenomenon of high perceptual variability-where individuals exhibit significant differences in emotion categorization even when viewing the same stimulus. Inspired by the similarity between ANNs and human perception, we hypothesize that facial expression samples that are ambiguous for ANN classifiers also elicit divergent perceptual judgments among human observers. To examine this hypothesis, we introduce a novel perceptual boundary sampling method to generate facial expression stimuli that lie along ANN decision boundaries. These ambiguous samples form the basis of the varEmotion dataset, constructed through large-scale human behavioral experiments. Our analysis reveals that these ANN-confusing stimuli also provoke heightened perceptual uncertainty in human participants, highlighting shared computational principles in emotion perception. Finally, by fine-tuning ANN representations using behavioral data, we achieve alignment between ANN predictions and both group-level and individual-level human perceptual patterns. Our findings establish a systematic link between ANN decision boundaries and human perceptual variability, offering new insights into personalized modeling of emotional interpretation.