# Py-Feat: Python Facial Expression Analysis Toolbox

arXiv: https://arxiv.org/abs/2104.03509 · HTML: https://mlsanity.com/paper/2104.03509/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 106 references that are in the map.
- First posted on arXiv 8 Apr 2021.

## Builds on (most cited first, 18 of 18)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [OpenAI Gym](https://mlsanity.com/paper/1606.01540/) (2016)
- [Stanza: A Python Natural Language Processing Toolkit for Many Human Languages](https://mlsanity.com/paper/2003.07082/) (2020)
- [AffectNet: A Database for Facial Expression, Valence, and Arousal Computing in the Wild](https://mlsanity.com/paper/1708.03985/) (2017)
- [Challenges in Representation Learning: A report on three machine learning contests](https://mlsanity.com/paper/1307.0414/) (2013)
- [WIDER FACE: A Face Detection Benchmark](https://mlsanity.com/paper/1511.06523/) (2015)
- [MobileFaceNets: Efficient CNNs for Accurate Real-Time Face Verification on Mobile Devices](https://mlsanity.com/paper/1804.07573/) (2018)
- [RetinaFace: Single-stage Dense Face Localisation in the Wild](https://mlsanity.com/paper/1905.00641/) (2019)
- [Machine Learning for Neuroimaging with Scikit-Learn](https://mlsanity.com/paper/1412.3919/) (2014)
- [From Facial Expression Recognition to Interpersonal Relation Prediction](https://mlsanity.com/paper/1609.06426/) (2016)
- [FaceBoxes: A CPU Real-time Face Detector with High Accuracy](https://mlsanity.com/paper/1708.05234/) (2017)
- [A Compact Embedding for Facial Expression Similarity](https://mlsanity.com/paper/1811.11283/) (2018)
- [PFLD: A Practical Facial Landmark Detector](https://mlsanity.com/paper/1902.10859/) (2019)
- [Deep Learning for Face Recognition: Pride or Prejudiced?](https://mlsanity.com/paper/1904.01219/) (2019)
- [EmotioNet Challenge: Recognition of facial expressions of emotion in the wild](https://mlsanity.com/paper/1703.01210/) (2017)
- [What your Facebook Profile Picture Reveals about your Personality](https://mlsanity.com/paper/1708.01292/) (2017)

## Built on by (most cited first, 30 of 33)

- [High-Fidelity and Freely Controllable Talking Head Video Generation](https://mlsanity.com/paper/2304.10168/) (2023)
- [Behaviour4All: in-the-wild Facial Behaviour Analysis Toolkit](https://mlsanity.com/paper/2409.17717/) (2024)
- [Textualized and Feature-based Models for Compound Multimodal Emotion Recognition in the Wild](https://mlsanity.com/paper/2407.12927/) (2024)
- [Tensor-based Emotion Editing in the StyleGAN Latent Space](https://mlsanity.com/paper/2205.06102/) (2022)
- [Learning Motion Refinement for Unsupervised Face Animation](https://mlsanity.com/paper/2310.13912/) (2023)
- [Transformer with Leveraged Masked Autoencoder for video-based Pain Assessment](https://mlsanity.com/paper/2409.05088/) (2024)
- [Visual and Textual Prompts in VLLMs for Enhancing Emotion Recognition](https://mlsanity.com/paper/2504.17224/) (2025)
- [OpenFace 3.0: A Lightweight Multitask System for Comprehensive Facial Behavior Analysis](https://mlsanity.com/paper/2506.02891/) (2025)
- [Using Positive Matching Contrastive Loss with Facial Action Units to mitigate bias in Facial Expression Recognition](https://mlsanity.com/paper/2303.04896/) (2023)
- [Solution for 10th Competition on Ambivalence/Hesitancy (AH) Video Recognition Challenge using Divergence-Based Multimodal Fusion](https://mlsanity.com/paper/2603.16939/) (2026)
- [Optimizing Facial Expressions of an Android Robot Effectively: a Bayesian Optimization Approach](https://mlsanity.com/paper/2301.05620/) (2023)
- [Beyond FACS: Data-driven Facial Expression Dictionaries, with Application to Predicting Autism](https://mlsanity.com/paper/2505.24679/) (2025)
- [SynPAIN: A Synthetic Dataset of Pain and Non-Pain Facial Expressions](https://mlsanity.com/paper/2507.19673/) (2025)
- [Interpretable Explainability in Facial Emotion Recognition and Gamification for Data Collection](https://mlsanity.com/paper/2211.04769/) (2022)
- [Integrating Holistic and Local Information to Estimate Emotional Reaction Intensity](https://mlsanity.com/paper/2305.05534/) (2023)
- [Behavior quantification as the missing link between fields: Tools for digital psychiatry and their role in the future of neurobiology](https://mlsanity.com/paper/2305.15385/) (2023)
- [Can we truly transfer an actor's genuine happiness to avatars? An investigation into virtual, real, posed and spontaneous faces](https://mlsanity.com/paper/2312.02128/) (2023)
- [The Meme Is the Message: Generative Memesis and AI Visuals in the 2024 USA Presidential Elections](https://mlsanity.com/paper/2411.00934/) (2024)
- [CAST-Phys: Contactless Affective States Through Physiological signals Database](https://mlsanity.com/paper/2507.06080/) (2025)
- [REVELIO -- Universal Multimodal Task Load Estimation for Cross-Domain Generalization](https://mlsanity.com/paper/2509.01642/) (2025)
- [Bitbox: Behavioral Imaging Toolbox for Computational Analysis of Behavior from Videos](https://mlsanity.com/paper/2512.17655/) (2025)
- [The Nonverbal Gap: Toward Affective Computer Vision for Safer and More Equitable Online Dating](https://mlsanity.com/paper/2603.26727/) (2026)
- [Interpretable facial dynamics as behavioral and perceptual traces of deepfakes](https://mlsanity.com/paper/2604.21760/) (2026)
- [3D Temporal Analysis for Autism Spectrum Disorder Screening During Attention Tasks](https://mlsanity.com/paper/2606.04836/) (2026)
- [Beyond Universality: The GCC-FER Dataset and Culture-Aware Adaptation for Dynamic Facial Expression Recognition](https://mlsanity.com/paper/2606.07063/) (2026)
- [STEER: Steerable Dyadic Head Avatars](https://mlsanity.com/paper/2607.23840/) (2026)
- [FIDAC: An Easy-to-use Pipeline to Extract and Interpret Interpersonal Distance From Video](https://mlsanity.com/paper/2607.25146/) (2026)
- [Emotion Intensity Matters: Generating Realistic Expressions in Virtual Humans with CVAEs](https://mlsanity.com/paper/2608.21697/) (2026)
- [Real-TurnTurk: A Multimodal Turkish Corpus for Turn-Taking Prediction](https://mlsanity.com/paper/2608.22071/) (2026)
- [TAMI: Temporally Aligned, Missingness-Aware, and Interpretable Multimodal Fusion for Mental Health Assessment in Older Adults with Mild Cognitive Impairment](https://mlsanity.com/paper/2608.30857/) (2026)

## Abstract (arXiv)

Studying facial expressions is a notoriously difficult endeavor. Recent advances in the field of affective computing have yielded impressive progress in automatically detecting facial expressions from pictures and videos. However, much of this work has yet to be widely disseminated in social science domains such as psychology. Current state of the art models require considerable domain expertise that is not traditionally incorporated into social science training programs. Furthermore, there is a notable absence of user-friendly and open-source software that provides a comprehensive set of tools and functions that support facial expression research. In this paper, we introduce Py-Feat, an open-source Python toolbox that provides support for detecting, preprocessing, analyzing, and visualizing facial expression data. Py-Feat makes it easy for domain experts to disseminate and benchmark computer vision models and also for end users to quickly process, analyze, and visualize face expression data. We hope this platform will facilitate increased use of facial expression data in human behavior research.