GANalyze: Toward Visual Definitions of Cognitive Image Properties
Not on the map. Why?
Abstract (arXiv)
We introduce a framework that uses Generative Adversarial Networks (GANs) to study cognitive properties like memorability, aesthetics, and emotional valence. These attributes are of interest because we do not have a concrete visual definition of what they entail. What does it look like for a dog to be more or less memorable? GANs allow us to generate a manifold of natural-looking images with fine-grained differences in their visual attributes. By navigating this manifold in directions that increase memorability, we can visualize what it looks like for a particular generated image to become more or less memorable. The resulting ``visual definitions" surface image properties (like ``object size") that may underlie memorability. Through behavioral experiments, we verify that our method indeed discovers image manipulations that causally affect human memory performance. We further demonstrate that the same framework can be used to analyze image aesthetics and emotional valence. Visit the GANalyze website at http://ganalyze.csail.mit.edu/.
Key facts. Cited by 193 papers in the map (data through 7 Oct 2026). First posted on arXiv 24 Jun 2019.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 58; 2022: 68; 2023: 35; 2024: 13; 2025: 8; 2026: 11.
193 papers.
Sort:
By area:
- 2021SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equationscited by 1,903
- 2021Designing an Encoder for StyleGAN Image Manipulationcited by 542
- 2022StyleGAN-XL: Scaling StyleGAN to Large Diverse Datasetscited by 489
- 2021Pivotal Tuning for Latent-based Editing of Real Imagescited by 427
- 2021GAN Inversion: A Surveycited by 366
- 2021ReStyle: A Residual-Based StyleGAN Encoder via Iterative Refinementcited by 248
- 2021HyperStyle: StyleGAN Inversion with HyperNetworks for Real Image Editingcited by 208
- 2023Interpreting CLIP's Image Representation via Text-Based Decompositioncited by 189
- 2022Latent Image Animator: Learning to Animate Images via Latent Space Navigationcited by 172
- 2021A Good Image Generator Is What You Need for High-Resolution Video Synthesiscited by 162
- 2021EditGAN: High-Precision Semantic Image Editingcited by 156
- 2021Explaining in Style: Training a GAN to explain a classifier in StyleSpacecited by 122
38 more
- 2021Paint by Wordcited by 107
- 2021Generative Models as a Data Source for Multiview Representation Learningcited by 104
- 2021Only a Matter of Style: Age Transformation Using a Style-Based Regression Modelcited by 92
- 2021Exploiting Spatial Dimensions of Latent in GAN for Real-time Image Editingcited by 91
- 2021HyperInverter: Improving StyleGAN Inversion via Hypernetworkcited by 85
- 2021HairCLIP: Design Your Hair by Text and Reference Imagecited by 79
- 2022Stitch it in Time: GAN-Based Facial Editing of Real Videoscited by 68
- 2023StyleDiffusion: Prompt-Embedding Inversion for Text-Based Editingcited by 68
- 2021A Latent Transformer for Disentangled Face Editing in Images and Videoscited by 67
- 2021Controllable and Compositional Generation with Latent-Space Energy-Based Modelscited by 64
- 2021Editing a classifier by rewriting its prediction rulescited by 64
- 2022Disentangling visual and written concepts in CLIPcited by 57
- 2021Using latent space regression to analyze and leverage compositionality in GANscited by 55
- 2021CAMPARI: Camera-Aware Decomposed Generative Neural Radiance Fieldscited by 55
- 2021Anycost GANs for Interactive Image Synthesis and Editingcited by 54
- 2022Third Time's the Charm? Image and Video Editing with StyleGAN3cited by 54
- 2021LatentCLR: A Contrastive Learning Approach for Unsupervised Discovery of Interpretable Directionscited by 52
- 2021Low-Rank Subspaces in GANscited by 52
- 2021Multimodal Image Synthesis and Editing: The Generative AI Eracited by 52
- 2021WarpedGANSpace: Finding non-linear RBF paths in GAN latent spacecited by 48
- 2023Rosetta Neurons: Mining the Common Units in a Model Zoocited by 46
- 2023NoiseCLR: A Contrastive Learning Approach for Unsupervised Discovery of Interpretable Directions in Diffusion Modelscited by 46
- 2021StyleMC: Multi-Channel Based Fast Text-Guided Image Generation and Manipulationcited by 40
- 2022Self-Distilled StyleGAN: Towards Generation from Internet Photoscited by 37
- 2022TransEditor: Transformer-Based Dual-Space GAN for Highly Controllable Facial Editingcited by 36
- 2022Text and Image Guided 3D Avatar Generation and Manipulationcited by 35
- 2022BlobGAN: Spatially Disentangled Scene Representationscited by 35
- 2021EigenGAN: Layer-Wise Eigen-Learning for GANscited by 34
- 2022Guided Co-Modulated GAN for 360deg Field of View Extrapolationcited by 32
- 2022GAN-generated Faces Detection: A Survey and New Perspectivescited by 29
- 2022Diffeomorphic Counterfactuals with Generative Modelscited by 29
- 2021StyleFusion: A Generative Model for Disentangling Spatial Segmentscited by 28
- 2022FlexIT: Towards Flexible Semantic Image Translationcited by 28
- 2021CoMoGAN: continuous model-guided image-to-image translationcited by 27
- 2021Do Not Escape From the Manifold: Discovering the Local Coordinates on the Latent Space of GANscited by 26
- 2021From Continuity to Editability: Inverting GANs with Consecutive Imagescited by 25
- 2021Improving GAN Equilibrium by Raising Spatial Awarenesscited by 22
- 2022Region-Based Semantic Factorization in GANscited by 22
50 most cited of 193 All 193 (JSON)
- 2026AI-Driven Neural Surrogates for In Silico Design of Cognitive-Affective Neuromodulation Targets23 Sep 2026
- 2026Learned Parametric Emotion Editing: Real-Time Affective Filtering for On-Device Social Media Video18 Sep 2026
- 2026RankT2I: A Submodular Framework for Discovering Interpretable and Diverse Semantics in Text-to-Image Models14 Aug 2026
- 2026Joint Flow Matching for Generator-Consistent Classification27 Jul 2026
- 2026CNS-Edit++: Category-Agnostic 3D Editing with Coupled Neural Shape Representation18 Jul 2026
- 2026Show Me Examples: Inferring Visual Concepts from Image Sets2 Jul 2026
- 2026Clinically-Grounded Counterfactual Reasoning for Medical Video Diagnosis26 May 2026
- 2026Explainable AI in Speaker Recognition -- Making Latent Representations Understandable25 Apr 2026
- 2026Scene Representation using 360deg Saliency Graph and its Application in Vision-based Indoor Navigation20 Mar 2026
- 2026CogBlender: Towards Continuous Cognitive Intervention in Text-to-Image Generation10 Mar 2026
- 2026How to Take a Memorable Picture? Empowering Users with Actionable Feedback25 Feb 2026
- 2025From Pixels to Feelings: Aligning MLLMs with Human Cognitive Perception of Images27 Nov 2025
38 more
- 2025SCALEX: Scalable Concept and Latent Exploration for Diffusion Models13 Nov 2025
- 2025LeapFactual: Reliable Visual Counterfactual Explanation Using Conditional Flow Matching16 Oct 2025
- 2025Be Tangential to Manifold: Discovering Riemannian Metric for Diffusion Models7 Oct 2025
- 2025CoBELa: Steering Transparent Generation via Concept Bottlenecks on Energy Landscapes11 Jul 2025
- 2025On the Role of Individual Differences in Current Approaches to Computational Image Aesthetics27 Feb 2025
- 2025DiffEx: Explaining a Classifier with Diffusion Models to Identify Microscopic Cellular Variations12 Feb 2025
- 2025Regressor-Guided Image Editing Shifts Emotion and Disengagement Timing in Social Media21 Jan 2025
- 2024Unforgettable Lessons from Forgettable Images: Intra-Class Memorability Matters in Computer Vision30 Dec 2024
- 2024Unsupervised Panoptic Interpretation of Latent Spaces in GANs Using Space-Filling Vector Quantization27 Oct 2024
- 2024Decoding Diffusion: A Scalable Framework for Unsupervised Analysis of Latent Space Biases and Representations Using Natural Language Prompts25 Oct 2024
- 2024Modeling Visual Memorability Assessment with Autoencoders Reveals Characteristics of Memorable Images19 Oct 2024
- 2024Unsupervised Representation Learning from Sparse Transformation Analysis7 Oct 2024
- 2024TACE: Tumor-Aware Counterfactual Explanations19 Sep 2024
- 2024The Gaussian Discriminant Variational Autoencoder (GdVAE): A Self-Explainable Model with Counterfactual Explanations19 Sep 2024
- 2024Multi-task SAR Image Processing via GAN-based Unsupervised Manipulation2 Aug 2024
- 2024SwinStyleformer is a favorable choice for image inversion19 Jun 2024
- 2024Make Me Happier: Evoking Emotions Through Image Diffusion Models13 Mar 2024
- 2024Counterfactual Image Editing7 Feb 2024
- 2024Closed-Loop Unsupervised Representation Disentanglement with β-VAE Distillation and Diffusion Probabilistic Feedback4 Feb 2024
- 2024CNS-Edit: 3D Shape Editing via Coupled Neural Shape Optimization4 Feb 2024
- 2023Spatial-Contextual Discrepancy Information Compensation for GAN Inversion12 Dec 2023
- 2023NoiseCLR: A Contrastive Learning Approach for Unsupervised Discovery of Interpretable Directions in Diffusion Models8 Dec 2023
- 2023Generative models for visualising abstract social processes: Guiding streetview image synthesis of StyleGAN2 with indices of deprivation1 Dec 2023
- 2023Seeing Eye to AI: Comparing Human Gaze and Model Attention in Video Memorability26 Nov 2023
- 2023Interpreting CLIP's Image Representation via Text-Based Decomposition9 Oct 2023
- 2023Trained Latent Space Navigation to Prevent Lack of Photorealism in Generated Images on Style-based Models2 Oct 2023
- 2023Locality-preserving Directions for Interpreting the Latent Space of Satellite Image GANs26 Sep 2023
- 2023In-Domain GAN Inversion for Faithful Reconstruction and Editability25 Sep 2023
- 2023Flow Factorized Representation Learning22 Sep 2023
- 2023Long-Term Ad Memorability: Understanding & Generating Memorable Ads1 Sep 2023
- 2023StylePrompter: All Styles Need Is Attention30 Jul 2023
- 2023LatentAugment: Data Augmentation via Guided Manipulation of GAN's Latent Space21 Jul 2023
- 2023Householder Projector for Unsupervised Latent Semantics Discovery16 Jul 2023
- 2023Generative Adversarial Networks for Dental Patient Identity Protection in Orthodontic Educational Imaging5 Jul 2023
- 2023Rosetta Neurons: Mining the Common Units in a Model Zoo15 Jun 2023
- 2023CLIPXPlore: Coupled CLIP and Shape Spaces for 3D Shape Exploration14 Jun 2023
- 2023Dynamically Masked Discriminator for Generative Adversarial Networks13 Jun 2023
- 2023TransMRSR: Transformer-based Self-Distilled Generative Prior for Brain MRI Super-Resolution11 Jun 2023
50 newest of 193 All 193 (JSON)
- 2023StyleDiffusion: Prompt-Embedding Inversion for Text-Based Editingcited by 68
- 2023NoiseCLR: A Contrastive Learning Approach for Unsupervised Discovery of Interpretable Directions in Diffusion Modelscited by 46
- 2024Make Me Happier: Evoking Emotions Through Image Diffusion Modelscited by 13
- 2024Closed-Loop Unsupervised Representation Disentanglement with β-VAE Distillation and Diffusion Probabilistic Feedbackcited by 9
- 2025LeapFactual: Reliable Visual Counterfactual Explanation Using Conditional Flow Matchingcited by 4
- 2025SCALEX: Scalable Concept and Latent Exploration for Diffusion Modelscited by 2
- 2024Decoding Diffusion: A Scalable Framework for Unsupervised Analysis of Latent Space Biases and Representations Using Natural Language Promptscited by 1
- 2024Unforgettable Lessons from Forgettable Images: Intra-Class Memorability Matters in Computer Visioncited by 1
- 2025Be Tangential to Manifold: Discovering Riemannian Metric for Diffusion Modelscited by 1
- 2025DiffEx: Explaining a Classifier with Diffusion Models to Identify Microscopic Cellular Variationscited by 0
- 2026Joint Flow Matching for Generator-Consistent Classificationcited by 0
- 2026Joint Flow Matching for Generator-Consistent Classification27 Jul 2026
- 2025SCALEX: Scalable Concept and Latent Exploration for Diffusion Models13 Nov 2025
- 2025LeapFactual: Reliable Visual Counterfactual Explanation Using Conditional Flow Matching16 Oct 2025
- 2025Be Tangential to Manifold: Discovering Riemannian Metric for Diffusion Models7 Oct 2025
- 2025DiffEx: Explaining a Classifier with Diffusion Models to Identify Microscopic Cellular Variations12 Feb 2025
- 2024Unforgettable Lessons from Forgettable Images: Intra-Class Memorability Matters in Computer Vision30 Dec 2024
- 2024Decoding Diffusion: A Scalable Framework for Unsupervised Analysis of Latent Space Biases and Representations Using Natural Language Prompts25 Oct 2024
- 2024Make Me Happier: Evoking Emotions Through Image Diffusion Models13 Mar 2024
- 2024Closed-Loop Unsupervised Representation Disentanglement with β-VAE Distillation and Diffusion Probabilistic Feedback4 Feb 2024
- 2023NoiseCLR: A Contrastive Learning Approach for Unsupervised Discovery of Interpretable Directions in Diffusion Models8 Dec 2023
- 2023StyleDiffusion: Prompt-Embedding Inversion for Text-Based Editing28 Mar 2023
- 2023Feature Unlearning for Pre-trained GANs and VAEscited by 15
- 2023Feature Unlearning for Pre-trained GANs and VAEs10 Mar 2023
- 2021SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equationscited by 1,903
- 2021Designing an Encoder for StyleGAN Image Manipulationcited by 542
- 2022StyleGAN-XL: Scaling StyleGAN to Large Diverse Datasetscited by 489
- 2021Pivotal Tuning for Latent-based Editing of Real Imagescited by 427
- 2021GAN Inversion: A Surveycited by 366
- 2021ReStyle: A Residual-Based StyleGAN Encoder via Iterative Refinementcited by 248
- 2021HyperStyle: StyleGAN Inversion with HyperNetworks for Real Image Editingcited by 208
- 2023Interpreting CLIP's Image Representation via Text-Based Decompositioncited by 189
- 2022Latent Image Animator: Learning to Animate Images via Latent Space Navigationcited by 172
- 2021A Good Image Generator Is What You Need for High-Resolution Video Synthesiscited by 162
- 2021EditGAN: High-Precision Semantic Image Editingcited by 156
- 2021Explaining in Style: Training a GAN to explain a classifier in StyleSpacecited by 122
12 most cited of 178 All 178 (JSON)
- 2026AI-Driven Neural Surrogates for In Silico Design of Cognitive-Affective Neuromodulation Targets23 Sep 2026
- 2026Learned Parametric Emotion Editing: Real-Time Affective Filtering for On-Device Social Media Video18 Sep 2026
- 2026RankT2I: A Submodular Framework for Discovering Interpretable and Diverse Semantics in Text-to-Image Models14 Aug 2026
- 2026CNS-Edit++: Category-Agnostic 3D Editing with Coupled Neural Shape Representation18 Jul 2026
- 2026Clinically-Grounded Counterfactual Reasoning for Medical Video Diagnosis26 May 2026
- 2026Explainable AI in Speaker Recognition -- Making Latent Representations Understandable25 Apr 2026
- 2026Scene Representation using 360deg Saliency Graph and its Application in Vision-based Indoor Navigation20 Mar 2026
- 2026CogBlender: Towards Continuous Cognitive Intervention in Text-to-Image Generation10 Mar 2026
- 2026How to Take a Memorable Picture? Empowering Users with Actionable Feedback25 Feb 2026
- 2025CoBELa: Steering Transparent Generation via Concept Bottlenecks on Energy Landscapes11 Jul 2025
- 2025On the Role of Individual Differences in Current Approaches to Computational Image Aesthetics27 Feb 2025
- 2025Regressor-Guided Image Editing Shifts Emotion and Disengagement Timing in Social Media21 Jan 2025
12 newest of 178 All 178 (JSON)