# A Compact Embedding for Facial Expression Similarity

arXiv: https://arxiv.org/abs/1811.11283 · HTML: https://mlsanity.com/paper/1811.11283/

## Key facts

- Cited by 41 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Nov 2018.

## Built on by (most cited first, 30 of 41)

- [Transformer-based Multimodal Information Fusion for Facial Expression Analysis](https://mlsanity.com/paper/2203.12367/) (2022)
- [FaceDancer: Pose- and Occlusion-Aware High Fidelity Face Swapping](https://mlsanity.com/paper/2210.10473/) (2022)
- [StyleSwap: Style-Based Generator Empowers Robust Face Swapping](https://mlsanity.com/paper/2209.13514/) (2022)
- [Py-Feat: Python Facial Expression Analysis Toolbox](https://mlsanity.com/paper/2104.03509/) (2021)
- [FSGANv2: Improved Subject Agnostic Face Swapping and Reenactment](https://mlsanity.com/paper/2202.12972/) (2022)
- [Leveraging Recent Advances in Deep Learning for Audio-Visual Emotion Recognition](https://mlsanity.com/paper/2103.09154/) (2021)
- [Prior Aided Streaming Network for Multi-task Affective Recognitionat the 2nd ABAW2 Competition](https://mlsanity.com/paper/2107.03708/) (2021)
- [ReSyncer: Rewiring Style-based Generator for Unified Audio-Visually Synced Facial Performer](https://mlsanity.com/paper/2408.03284/) (2024)
- [VEATIC: Video-based Emotion and Affect Tracking in Context Dataset](https://mlsanity.com/paper/2309.06745/) (2023)
- [High-dimensional separability for one- and few-shot learning](https://mlsanity.com/paper/2106.15416/) (2021)
- [Reinforced Disentanglement for Face Swapping without Skip Connection](https://mlsanity.com/paper/2307.07928/) (2023)
- [DeepFace-Attention: Multimodal Face Biometrics for Attention Estimation with Application to e-Learning](https://mlsanity.com/paper/2408.05523/) (2024)
- [FlowFace: Semantic Flow-guided Shape-aware Face Swapping](https://mlsanity.com/paper/2212.02797/) (2022)
- [A View From Somewhere: Human-Centric Face Representations](https://mlsanity.com/paper/2303.17176/) (2023)
- [Unsupervised Facial Expression Representation Learning with Contrastive Local Warping](https://mlsanity.com/paper/2303.09034/) (2023)
- [FreeAvatar: Robust 3D Facial Animation Transfer by Learning an Expression Foundation Model](https://mlsanity.com/paper/2409.13180/) (2024)
- [Which CNNs and Training Settings to Choose for Action Unit Detection? A Study Based on a Large-Scale Dataset](https://mlsanity.com/paper/2111.08320/) (2021)
- [QCS: Feature Refining from Quadruplet Cross Similarity for Facial Expression Recognition](https://mlsanity.com/paper/2411.01988/) (2024)
- [Affect Expression Behaviour Analysis in the Wild using Consensual Collaborative Training](https://mlsanity.com/paper/2107.05736/) (2021)
- [Choose Settings Carefully: Comparing Action Unit detection at Different Settings Using a Large-Scale Dataset](https://mlsanity.com/paper/2111.08324/) (2021)
- [FlowFace++: Explicit Semantic Flow-supervised End-to-End Face Swapping](https://mlsanity.com/paper/2306.12686/) (2023)
- [RID-TWIN: An end-to-end pipeline for automatic face de-identification in videos](https://mlsanity.com/paper/2403.10058/) (2024)
- [Towards a General Deep Feature Extractor for Facial Expression Recognition](https://mlsanity.com/paper/2201.07781/) (2022)
- [Facial Action Units Detection Aided by Global-Local Expression Embedding](https://mlsanity.com/paper/2210.13718/) (2022)
- [Rank-based No-reference Quality Assessment for Face Swapping](https://mlsanity.com/paper/2406.01884/) (2024)
- [Beyond Inference Intervention: Identity-Decoupled Diffusion for Face Anonymization](https://mlsanity.com/paper/2510.24213/) (2025)
- [ARPGNet: Appearance- and Relation-aware Parallel Graph Attention Fusion Network for Facial Expression Recognition](https://mlsanity.com/paper/2511.22188/) (2025)
- [Alignment-Process-Outcome: Rethinking How AIs and Humans Collaborate](https://mlsanity.com/paper/2603.08017/) (2026)
- [Emotion Transfer Using Vector-Valued Infinite Task Learning](https://mlsanity.com/paper/2102.05075/) (2021)
- [PatentNet: A Large-Scale Incomplete Multiview, Multimodal, Multilabel Industrial Goods Image Database](https://mlsanity.com/paper/2106.12139/) (2021)

## Abstract (arXiv)

Most of the existing work on automatic facial expression analysis focuses on discrete emotion recognition, or facial action unit detection. However, facial expressions do not always fall neatly into pre-defined semantic categories. Also, the similarity between expressions measured in the action unit space need not correspond to how humans perceive expression similarity. Different from previous work, our goal is to describe facial expressions in a continuous fashion using a compact embedding space that mimics human visual preferences. To achieve this goal, we collect a large-scale faces-in-the-wild dataset with human annotations in the form: Expressions A and B are visually more similar when compared to expression C, and use this dataset to train a neural network that produces a compact (16-dimensional) expression embedding. We experimentally demonstrate that the learned embedding can be successfully used for various applications such as expression retrieval, photo album summarization, and emotion recognition. We also show that the embedding learned using the proposed dataset performs better than several other embeddings learned using existing emotion or action unit datasets.