# Facial Expression Recognition with Controlled Privacy Preservation and Feature Compensation

arXiv: https://arxiv.org/abs/2412.00277 · HTML: https://mlsanity.com/paper/2412.00277/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 63 references that are in the map.
- First posted on arXiv 29 Nov 2024.

## Builds on (most cited first, 25 of 25)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [The Kinetics Human Action Video Dataset](https://mlsanity.com/paper/1705.06950/) (2017)
- [A Closer Look at Spatiotemporal Convolutions for Action Recognition](https://mlsanity.com/paper/1711.11248/) (2017)
- [All are Worth Words: A ViT Backbone for Diffusion Models](https://mlsanity.com/paper/2209.12152/) (2022)
- [Deep Facial Expression Recognition: A Survey](https://mlsanity.com/paper/1804.08348/) (2018)
- [DFEW: A Large-Scale Database for Recognizing Dynamic Facial Expressions in the Wild](https://mlsanity.com/paper/2008.05924/) (2020)
- [VideoFlow: Exploiting Temporal Cues for Multi-frame Optical Flow Estimation](https://mlsanity.com/paper/2303.08340/) (2023)
- [On the Reconstruction of Face Images from Deep Face Templates](https://mlsanity.com/paper/1703.00832/) (2017)
- [Few-shot Object Counting with Similarity-Aware Feature Enhancement](https://mlsanity.com/paper/2201.08959/) (2022)
- [SPAct: Self-supervised Privacy Preservation for Action Recognition](https://mlsanity.com/paper/2203.15205/) (2022)
- [Intensity-Aware Loss for Dynamic Facial Expression Recognition in the Wild](https://mlsanity.com/paper/2208.10335/) (2022)
- [Factorized Higher-Order CNNs with an Application to Spatio-Temporal Emotion Estimation](https://mlsanity.com/paper/1906.06196/) (2019)
- [MobileFaceSwap: A Lightweight Framework for Video Face Swapping](https://mlsanity.com/paper/2201.03808/) (2022)
- [Privacy-Preserving Face Recognition with Learnable Privacy Budgets in Frequency Domain](https://mlsanity.com/paper/2207.07316/) (2022)
- [Connecting Pixels to Privacy and Utility: Automatic Redaction of Private Information in Images](https://mlsanity.com/paper/1712.01066/) (2017)
- [Privacy-Preserving Face Recognition Using Trainable Feature Subtraction](https://mlsanity.com/paper/2403.12457/) (2024)
- [TeD-SPAD: Temporal Distinctiveness for Self-supervised Privacy-preservation for video Anomaly Detection](https://mlsanity.com/paper/2308.11072/) (2023)
- [Black-Box Face Recovery from Identity Features](https://mlsanity.com/paper/2007.13635/) (2020)
- [Privacy-Preserving Face Recognition Using Random Frequency Components](https://mlsanity.com/paper/2308.10461/) (2023)
- [Color Shift Estimation-and-Correction for Image Enhancement](https://mlsanity.com/paper/2405.17725/) (2024)
- [VGAN-Based Image Representation Learning for Privacy-Preserving Facial Expression Recognition](https://mlsanity.com/paper/1803.07100/) (2018)
- [Optical Flow Techniques for Facial Expression Analysis -- a Practical Evaluation Study](https://mlsanity.com/paper/1904.11592/) (2019)
- [Orientation-conditioned Facial Texture Mapping for Video-based Facial Remote Photoplethysmography Estimation](https://mlsanity.com/paper/2404.09378/) (2024)
- [An adversarial learning framework for preserving users' anonymity in face-based emotion recognition](https://mlsanity.com/paper/2001.06103/) (2020)

## Built on by (most cited first, 1 of 1)

- [Knowledge Priors for Identity-Disentangled Open-Set Privacy-Preserving Video FER](https://mlsanity.com/paper/2603.21387/) (2026)

## Abstract (arXiv)

Facial expression recognition (FER) systems raise significant privacy concerns due to the potential exposure of sensitive identity information. This paper presents a study on removing identity information while preserving FER capabilities. Drawing on the observation that low-frequency components predominantly contain identity information and high-frequency components capture expression, we propose a novel two-stream framework that applies privacy enhancement to each component separately. We introduce a controlled privacy enhancement mechanism to optimize performance and a feature compensator to enhance task-relevant features without compromising privacy. Furthermore, we propose a novel privacy-utility trade-off, providing a quantifiable measure of privacy preservation efficacy in closed-set FER tasks. Extensive experiments on the benchmark CREMA-D dataset demonstrate that our framework achieves 78.84% recognition accuracy with a privacy (facial identity) leakage ratio of only 2.01%, highlighting its potential for secure and reliable video-based FER applications.