# AffectGAN: Affect-Based Generative Art Driven by Semantics

arXiv: https://arxiv.org/abs/2109.14845 · HTML: https://mlsanity.com/paper/2109.14845/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 29 references that are in the map.
- First posted on arXiv 30 Sep 2021.

## Builds on (most cited first, 14 of 14)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Taming Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2012.09841/) (2020)
- [Generative Adversarial Text to Image Synthesis](https://mlsanity.com/paper/1605.05396/) (2016)
- [AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networks](https://mlsanity.com/paper/1711.10485/) (2017)
- [StackGAN: Text to Photo-realistic Image Synthesis with Stacked Generative Adversarial Networks](https://mlsanity.com/paper/1612.03242/) (2016)
- [Generating Images from Captions with Attention](https://mlsanity.com/paper/1511.02793/) (2015)
- [Object-driven Text-to-Image Synthesis via Adversarial Training](https://mlsanity.com/paper/1902.10740/) (2019)
- [Paint by Word](https://mlsanity.com/paper/2103.10951/) (2021)
- [ArtEmis: Affective Language for Visual Art](https://mlsanity.com/paper/2101.07396/) (2021)
- [Generating images from caption and vice versa via CLIP-Guided Generative Latent Space Search](https://mlsanity.com/paper/2102.01645/) (2021)
- [The Pixels and Sounds of Emotion: General-Purpose Representations of Arousal in Games](https://mlsanity.com/paper/2101.10706/) (2021)
- [From Pixels to Affect: A Study on Games and Player Experience](https://mlsanity.com/paper/1907.02288/) (2019)

## Built on by (most cited first, 10 of 10)

- [VQGAN-CLIP: Open Domain Image Generation and Editing with Natural Language Guidance](https://mlsanity.com/paper/2204.08583/) (2022)
- [A Comprehensive Survey on Applications of Transformers for Deep Learning Tasks](https://mlsanity.com/paper/2306.07303/) (2023)
- [RePrompt: Automatic Prompt Editing to Refine AI-Generative Art Towards Precise Expressions](https://mlsanity.com/paper/2302.09466/) (2023)
- [Cut the CARP: Fishing for zero-shot story evaluation](https://mlsanity.com/paper/2110.03111/) (2021)
- [Affective Game Computing: A Survey](https://mlsanity.com/paper/2309.14104/) (2023)
- [Seeding Diversity into AI Art](https://mlsanity.com/paper/2205.00804/) (2022)
- [Affect-Conditioned Image Generation](https://mlsanity.com/paper/2302.09742/) (2023)
- [AIM-Bench: Benchmarking and Improving Affective Image Manipulation via Fine-Grained Hierarchical Control](https://mlsanity.com/paper/2604.10454/) (2026)
- [Threshold Designer Adaptation: Improved Adaptation for Designers in Co-creative Systems](https://mlsanity.com/paper/2205.09269/) (2022)
- [EmoScene: A Dual-space Dataset for Controllable Affective Image Generation](https://mlsanity.com/paper/2604.00933/) (2026)

## Abstract (arXiv)

This paper introduces a novel method for generating artistic images that express particular affective states. Leveraging state-of-the-art deep learning methods for visual generation (through generative adversarial networks), semantic models from OpenAI, and the annotated dataset of the visual art encyclopedia WikiArt, our AffectGAN model is able to generate images based on specific or broad semantic prompts and intended affective outcomes. A small dataset of 32 images generated by AffectGAN is annotated by 50 participants in terms of the particular emotion they elicit, as well as their quality and novelty. Results show that for most instances the intended emotion used as a prompt for image generation matches the participants' responses. This small-scale study brings forth a new vision towards blending affective computing with computational creativity, enabling generative systems with intentionality in terms of the emotions they wish their output to elicit.