mlsanityData to 14 Oct 2026

SEREP: Semantic Facial Expression Representation for Robust In-the-Wild Capture and Retargeting

arXiv 2412.14371 ↗First posted 18 Dec 2024cs.CV cs.GR cs.LGcited by 4

Not on the map. Why?

Abstract (arXiv)

Monocular facial performance capture in-the-wild is challenging due to varied capture conditions, face shapes, and expressions. Most current methods rely on linear 3D Morphable Models, which represent facial expressions independently of identity at the vertex displacement level. We propose SEREP (Semantic Expression Representation), a model that disentangles expression from identity at the semantic level. We start by learning an expression representation from high-quality 3D data of unpaired facial expressions. Then, we train a model to predict expression from monocular images relying on a novel semi-supervised scheme using low quality synthetic data. In addition, we introduce MultiREX, a benchmark addressing the lack of evaluation resources for the expression capture task. Our experiments show that SEREP outperforms state-of-the-art methods, capturing challenging expressions and transferring them to new identities.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 42 of its 59 references that are in the map. First posted on arXiv 18 Dec 2024.

Builds on

42 of 59 references.

References · 42

  1. 2022A ConvNet for the 2020scited by 3,579
  2. 2016Identity Mappings in Deep Residual Networkscited by 1,965
  3. 2014Unsupervised Domain Adaptation by Backpropagationcited by 1,917
  4. 2018Group Normalizationcited by 1,524
  5. 2018Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representationscited by 941
  6. 2019Accurate 3D Face Reconstruction with Weakly-Supervised Learning: From Single Image to Image Setcited by 482
  7. 2020Learning an Animatable Detailed 3D Face Model from In-The-Wild Imagescited by 477
  8. 2020State of the Art on Neural Renderingcited by 278
  9. 20193D Morphable Face Models -- Past, Present and Futurecited by 270
  10. 2018Generating 3D faces using Convolutional Mesh Autoencoderscited by 268
  11. 2022EMOCA: Emotion Driven Monocular Face Capture and Animationcited by 255
  12. 2022CelebV-HQ: A Large-Scale Video Facial Attributes Datasetcited by 250
30 more
  1. 2021Fake It Till You Make It: Face analysis in the wild using synthetic data alonecited by 200
  2. 2020FaceScape: a Large-scale High Quality 3D Face Dataset and Detailed Riggable 3D Face Predictioncited by 196
  3. 2024VASA-1: Lifelike Audio-Driven Talking Faces Generated in Real Timecited by 187
  4. 2019Learning to Regress 3D Face Shape and Expression from an Image without 3D Supervisioncited by 176
  5. 2022Towards Metrical Reconstruction of Human Facescited by 166
  6. 2018Look at Boundary: A Boundary-Aware Face Alignment Algorithmcited by 138
  7. 2022MegaPortraits: One-shot Megapixel Neural Head Avatarscited by 131
  8. 2020Differentiable Rendering: A Surveycited by 123
  9. 2023Emotional Speech-Driven Animation with Content-Emotion Disentanglementcited by 98
  10. 2020Learning Formation of Physically-Based Face Attributescited by 94
  11. 2016Learning Dense Correspondence via 3D-guided Cycle Consistencycited by 91
  12. 2019Neural 3D Morphable Models: Spiral Convolutional Networks for 3D Shape Representation Learning and Generationcited by 90
  13. 2022Multiface: A Dataset for Neural Face Renderingcited by 87
  14. 2024SMIRK: 3D Facial Expressions through Analysis-by-Neural-Synthesiscited by 84
  15. 2022Learning Neural Parametric Head Modelscited by 73
  16. 2020Measuring Disentanglement: A Review of Metricscited by 70
  17. 20223D face reconstruction with dense landmarkscited by 70
  18. 2023A Hierarchical Representation Network for Accurate and Detailed Face Reconstruction from In-The-Wild Imagescited by 43
  19. 2023STAR Loss: Reducing Semantic Ambiguity in Facial Landmark Detectioncited by 43
  20. 2024Media2Face: Co-speech Facial Animation Generation With Multi-Modality Guidancecited by 39
  21. 20233D Face Reconstruction with the Geometric Guidance of Facial Part Segmentationcited by 38
  22. 2023MonoNPHM: Dynamic Head Reconstruction from Monocular Videoscited by 35
  23. 2023Instant Multi-View Head Capture through Learnable Registrationcited by 34
  24. 2022REALY: Rethinking the Evaluation of 3D Face Reconstructioncited by 33
  25. 2024Look Ma, no markers: holistic performance capture without the hasslecited by 23
  26. 2023HiFace: High-Fidelity 3D Face Reconstruction by Learning Static and Dynamic Detailscited by 22
  27. 2023MoSAR: Monocular Semi-Supervised Model for Avatar Reconstruction using Differentiable Shadingcited by 18
  28. 2024SPARK: Self-supervised Personalized Real-time Monocular Face Capturecited by 6
  29. 2024FaceLift: Semi-supervised 3D Facial Landmark Localizationcited by 2
  30. 2020Unsupervised Performance Analysis of 3D Face Alignment with a Statistically Robust Confidence Testcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 3.

4 papers.

  1. 2026SemanticFace: Semantic Facial Action Estimation via Semantic Distillation in Interpretable Spacecited by 2
  2. 2025FIELDS: Face reconstruction with accurate Inference of Expression using Learning with Direct Supervisioncited by 0
  3. 2026KM-Speaker: Keypoint-Based Style Control for High-Quality Speech-Driven 3D Facial Animation and Dialogue Localizationcited by 0
  4. 2026FaceSnap: Real-Time Personalized Lightstage Facial Performance Capturecited by 0