# Facial Expression Recognition Under Partial Occlusion from Virtual Reality Headsets based on Transfer Learning

arXiv: https://arxiv.org/abs/2008.05563 · HTML: https://mlsanity.com/paper/2008.05563/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Aug 2020.

## Built on by (most cited first, 7 of 7)

- [Teacher-Student Training and Triplet Loss to Reduce the Effect of Drastic Face Occlusion](https://mlsanity.com/paper/2111.10561/) (2021)
- [In the Blink of an Eye: Event-based Emotion Recognition](https://mlsanity.com/paper/2310.04043/) (2023)
- [Apprenticeship-Inspired Elegance: Synergistic Knowledge Distillation Empowers Spiking Neural Networks for Efficient Single-Eye Emotion Recognition](https://mlsanity.com/paper/2407.09521/) (2024)
- [EmojiHeroVR: A Study on Facial Expression Recognition under Partial Occlusion from Head-Mounted Displays](https://mlsanity.com/paper/2410.03331/) (2024)
- [Region attention and graph embedding network for occlusion objective class-based micro-expression recognition](https://mlsanity.com/paper/2107.05904/) (2021)
- [FERT: Real-Time Facial Expression Recognition with Short-Range FMCW Radar](https://mlsanity.com/paper/2411.11619/) (2024)
- [Unimodal and Multimodal Static Facial Expression Recognition for Virtual Reality Users with EmoHeVRDB](https://mlsanity.com/paper/2412.11306/) (2024)

## Abstract (arXiv)

Facial expressions of emotion are a major channel in our daily communications, and it has been subject of intense research in recent years. To automatically infer facial expressions, convolutional neural network based approaches has become widely adopted due to their proven applicability to Facial Expression Recognition (FER) task.On the other hand Virtual Reality (VR) has gained popularity as an immersive multimedia platform, where FER can provide enriched media experiences. However, recognizing facial expression while wearing a head-mounted VR headset is a challenging task due to the upper half of the face being completely occluded. In this paper we attempt to overcome these issues and focus on facial expression recognition in presence of a severe occlusion where the user is wearing a head-mounted display in a VR setting. We propose a geometric model to simulate occlusion resulting from a Samsung Gear VR headset that can be applied to existing FER datasets. Then, we adopt a transfer learning approach, starting from two pretrained networks, namely VGG and ResNet. We further fine-tune the networks on FER+ and RAF-DB datasets. Experimental results show that our approach achieves comparable results to existing methods while training on three modified benchmark datasets that adhere to realistic occlusion resulting from wearing a commodity VR headset. Code for this paper is available at: https://github.com/bita-github/MRP-FER