# Neuromorphic Facial Analysis with Cross-Modal Supervision arXiv: https://arxiv.org/abs/2409.10213 ยท HTML: https://mlsanity.com/paper/2409.10213/ ## Key facts - Cited by 7 papers in the map (data through 14 Oct 2026). - Builds on 29 of its 76 references that are in the map. - First posted on arXiv 16 Sep 2024. ## Builds on (most cited first, 29 of 29) - [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015) - [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017) - [Event-based Vision: A Survey](https://mlsanity.com/paper/1904.08405/) (2019) - [How far are we from solving the 2D & 3D Face Alignment problem? (and a dataset of 230,000 3D facial landmarks)](https://mlsanity.com/paper/1703.07332/) (2017) - [Generating 3D faces using Convolutional Mesh Autoencoders](https://mlsanity.com/paper/1807.10267/) (2018) - [Capture, Learning, and Synthesis of 3D Speaking Styles](https://mlsanity.com/paper/1905.03079/) (2019) - [Nonlinear 3D Face Morphable Model](https://mlsanity.com/paper/1804.03786/) (2018) - [GANFIT: Generative Adversarial Network Fitting for High Fidelity 3D Face Reconstruction](https://mlsanity.com/paper/1902.05978/) (2019) - [Event-based Moving Object Detection and Tracking](https://mlsanity.com/paper/1803.04523/) (2018) - [MPIIGaze: Real-World Dataset and Deep Appearance-Based Gaze Estimation](https://mlsanity.com/paper/1711.09017/) (2017) - [Regressing Robust and Discriminative 3D Morphable Models with a very Deep Neural Network](https://mlsanity.com/paper/1612.04904/) (2016) - [Neural 3D Morphable Models: Spiral Convolutional Networks for 3D Shape Representation Learning and Generation](https://mlsanity.com/paper/1905.02876/) (2019) - [ImFace: A Nonlinear 3D Morphable Face Model with Implicit Neural Representations](https://mlsanity.com/paper/2203.14510/) (2022) - [Temporal Binary Representation for Event-Based Action Recognition](https://mlsanity.com/paper/2010.08946/) (2020) - [Survey on RGB, 3D, Thermal, and Multimodal Approaches for Facial Expression Recognition: History, Trends, and Affect-related Applications](https://mlsanity.com/paper/1606.03237/) (2016) - [MVF-Net: Multi-View 3D Face Morphable Model Regression](https://mlsanity.com/paper/1904.04473/) (2019) - [DART: Distribution Aware Retinal Transform for Event-based Cameras](https://mlsanity.com/paper/1710.10800/) (2017) - [Exploring Event Camera-based Odometry for Planetary Robots](https://mlsanity.com/paper/2204.05880/) (2022) - [Multilinear Wavelets: A Statistical Shape Space for Human Faces](https://mlsanity.com/paper/1401.2818/) (2014) - [Gaussian Process Morphable Models](https://mlsanity.com/paper/1603.07254/) (2016) - [ExpNet: Landmark-Free, Deep, 3D Facial Expressions](https://mlsanity.com/paper/1802.00542/) (2018) - [Motion Robust High-Speed Light-Weighted Object Detection With Event Camera](https://mlsanity.com/paper/2208.11602/) (2022) - [Neuromorphic Event-based Facial Expression Recognition](https://mlsanity.com/paper/2304.06351/) (2023) - [Event-based Face Detection and Tracking in the Blink of an Eye](https://mlsanity.com/paper/1803.10106/) (2018) - [Event-based attention and tracking on neuromorphic hardware](https://mlsanity.com/paper/1907.04060/) (2019) - [Neuromorphic Face Analysis: a Survey](https://mlsanity.com/paper/2402.11631/) (2024) - [The Florence 4D Facial Expression Dataset](https://mlsanity.com/paper/2210.16807/) (2022) - [Generating Multiple 4D Expression Transitions by Learning Face Landmark Trajectories](https://mlsanity.com/paper/2208.00050/) (2022) - [Neuromorphic Valence and Arousal Estimation](https://mlsanity.com/paper/2401.16058/) (2024) ## Built on by (most cited first, 7 of 7) - [Exploring Spatial-Temporal Dynamics in Event-based Facial Micro-Expression Analysis](https://mlsanity.com/paper/2508.11988/) (2025) - [CS3D: An Efficient Facial Expression Recognition via Event Vision](https://mlsanity.com/paper/2512.09592/) (2025) - [Spike-TBR: a Noise Resilient Neuromorphic Event Representation](https://mlsanity.com/paper/2506.04817/) (2025) - [Event-based Facial Keypoint Alignment via Cross-Modal Fusion Attention and Self-Supervised Multi-Event Representation Learning](https://mlsanity.com/paper/2509.24968/) (2025) - [EventFace: Event-Based Face Recognition via Structure-Driven Spatiotemporal Modeling](https://mlsanity.com/paper/2604.06782/) (2026) - [CDER-SME: A Cross-Device Event-RGB Micro-Expression Dataset under Multi-Level Stress Induction](https://mlsanity.com/paper/2606.20715/) (2026) - [Emo-DVS: A Multimodal Benchmark for Privacy-Aware Emotion Recognition with Event Cameras](https://mlsanity.com/paper/2609.06928/) (2026) ## Abstract (arXiv) Traditional approaches for analyzing RGB frames are capable of providing a fine-grained understanding of a face from different angles by inferring emotions, poses, shapes, landmarks. However, when it comes to subtle movements standard RGB cameras might fall behind due to their latency, making it hard to detect micro-movements that carry highly informative cues to infer the true emotions of a subject. To address this issue, the usage of event cameras to analyze faces is gaining increasing interest. Nonetheless, all the expertise matured for RGB processing is not directly transferrable to neuromorphic data due to a strong domain shift and intrinsic differences in how data is represented. The lack of labeled data can be considered one of the main causes of this gap, yet gathering data is harder in the event domain since it cannot be crawled from the web and labeling frames should take into account event aggregation rates and the fact that static parts might not be visible in certain frames. In this paper, we first present FACEMORPHIC, a multimodal temporally synchronized face dataset comprising both RGB videos and event streams. The data is labeled at a video level with facial Action Units and also contains streams collected with a variety of applications in mind, ranging from 3D shape estimation to lip-reading. We then show how temporal synchronization can allow effective neuromorphic face analysis without the need to manually annotate videos: we instead leverage cross-modal supervision bridging the domain gap by representing face shapes in a 3D space.