# A Unified and Interpretable Emotion Representation and Expression Generation arXiv: https://arxiv.org/abs/2404.01243 ยท HTML: https://mlsanity.com/paper/2404.01243/ ## Key facts - Cited by 11 papers in the map (data through 14 Oct 2026). - Builds on 34 of its 50 references that are in the map. - First posted on arXiv 1 Apr 2024. ## Builds on (most cited first, 30 of 34) - [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021) - [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020) - [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021) - [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014) - [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020) - [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020) - [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015) - [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022) - [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022) - [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022) - [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021) - [GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models](https://mlsanity.com/paper/2112.10741/) (2021) - [DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation](https://mlsanity.com/paper/2208.12242/) (2022) - [An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion](https://mlsanity.com/paper/2208.01618/) (2022) - [Scaling Autoregressive Models for Content-Rich Text-to-Image Generation](https://mlsanity.com/paper/2206.10789/) (2022) - [Multi-Concept Customization of Text-to-Image Diffusion](https://mlsanity.com/paper/2212.04488/) (2022) - [StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translation](https://mlsanity.com/paper/1711.09020/) (2017) - [eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoisers](https://mlsanity.com/paper/2211.01324/) (2022) - [Vector Quantized Diffusion Model for Text-to-Image Synthesis](https://mlsanity.com/paper/2111.14822/) (2021) - [Compositional Visual Generation with Composable Diffusion Models](https://mlsanity.com/paper/2206.01714/) (2022) - [AffectNet: A Database for Facial Expression, Valence, and Arousal Computing in the Wild](https://mlsanity.com/paper/1708.03985/) (2017) - [Aligning Text-to-Image Models using Human Feedback](https://mlsanity.com/paper/2302.12192/) (2023) - [Training-Free Structured Diffusion Guidance for Compositional Text-to-Image Synthesis](https://mlsanity.com/paper/2212.05032/) (2022) - [GAN Inversion: A Survey](https://mlsanity.com/paper/2101.05278/) (2021) - [GANimation: Anatomically-aware Facial Animation from a Single Image](https://mlsanity.com/paper/1807.09251/) (2018) - [Uncovering the Disentanglement Capability in Text-to-Image Diffusion Models](https://mlsanity.com/paper/2212.08698/) (2022) - [Learning Multi-dimensional Edge Feature-based AU Relation Graph for Facial Action Unit Recognition](https://mlsanity.com/paper/2205.01782/) (2022) - [3DDesigner: Towards Photorealistic 3D Object Generation and Editing with Text-guided Diffusion Models](https://mlsanity.com/paper/2211.14108/) (2022) - [ICface: Interpretable and Controllable Face Reenactment Using GANs](https://mlsanity.com/paper/1904.01909/) (2019) - [Covariance Pooling For Facial Expression Recognition](https://mlsanity.com/paper/1805.04855/) (2018) ## Built on by (most cited first, 11 of 11) - [X-NeMo: Expressive Neural Motion Reenactment via Disentangled Latent Attention](https://mlsanity.com/paper/2507.23143/) (2025) - [Towards Localized Fine-Grained Control for Facial Expression Generation](https://mlsanity.com/paper/2407.20175/) (2024) - [Affective Image Editing: Shaping Emotional Factors via Text Descriptions](https://mlsanity.com/paper/2505.18699/) (2025) - [EmojiDiff: Advanced Facial Expression Control with High Identity Preservation in Portrait Generation](https://mlsanity.com/paper/2412.01254/) (2024) - [What Can I Edit? Open-Ended Strategy Discovery and the Emotion Editability Landscape](https://mlsanity.com/paper/2607.23920/) (2026) - [Temporal and Spatial Feature Fusion Framework for Dynamic Micro Expression Recognition](https://mlsanity.com/paper/2505.16372/) (2025) - [ID-Consistent, Precise Expression Generation with Blendshape-Guided Diffusion](https://mlsanity.com/paper/2510.04706/) (2025) - [EmoScene: A Dual-space Dataset for Controllable Affective Image Generation](https://mlsanity.com/paper/2604.00933/) (2026) - [Emotion Knowledge Enhancement for Vision Large Language Models: A Self-Verification Approach for High-Quality Emotion Instruction Data Generation](https://mlsanity.com/paper/2505.18168/) (2025) - [Instant Expressive Gaussian Head Avatars at Over 100 FPS](https://mlsanity.com/paper/2512.16893/) (2025) - [Barycentric Guidance: Turning Foundation Image Editors into Continuous Affective Controllers](https://mlsanity.com/paper/2610.16892/) (2026) ## Abstract (arXiv) Canonical emotions, such as happy, sad, and fearful, are easy to understand and annotate. However, emotions are often compound, e.g. happily surprised, and can be mapped to the action units (AUs) used for expressing emotions, and trivially to the canonical ones. Intuitively, emotions are continuous as represented by the arousal-valence (AV) model. An interpretable unification of these four modalities - namely, Canonical, Compound, AUs, and AV - is highly desirable, for a better representation and understanding of emotions. However, such unification remains to be unknown in the current literature. In this work, we propose an interpretable and unified emotion model, referred as C2A2. We also develop a method that leverages labels of the non-unified models to annotate the novel unified one. Finally, we modify the text-conditional diffusion models to understand continuous numbers, which are then used to generate continuous expressions using our unified emotion model. Through quantitative and qualitative experiments, we show that our generated images are rich and capture subtle expressions. Our work allows a fine-grained generation of expressions in conjunction with other textual inputs and offers a new label space for emotions at the same time.