mlsanityData to 14 Oct 2026

General Facial Representation Learning in a Visual-Linguistic Manner

arXiv 2112.03109 ↗First posted 6 Dec 2021cs.CV cs.CLcited by 195

Not on the map. Why?

Cited by 30 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

How to learn a universal facial representation that boosts all face analysis tasks? This paper takes one step toward this goal. In this paper, we study the transfer performance of pre-trained models on face analysis tasks and introduce a framework, called FaRL, for general Facial Representation Learning in a visual-linguistic manner. On one hand, the framework involves a contrastive loss to learn high-level semantic meaning from image-text pairs. On the other hand, we propose exploring low-level information simultaneously to further enhance the face representation, by adding a masked image modeling. We perform pre-training on LAION-FACE, a dataset containing large amount of face image-text pairs, and evaluate the representation capability on multiple downstream tasks. We show that FaRL achieves better transfer performance compared with previous pre-trained models. We also verify its superiority in the low-data regime. More importantly, our model surpasses the state-of-the-art methods on face analysis tasks including face parsing and face alignment.

Key facts. Cited by 195 papers in the map (data through 14 Oct 2026). Builds on 75 of its 116 references that are in the map. First posted on arXiv 6 Dec 2021.

Builds on

75 of 116 references.

References · 75

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  4. 2020Language Models are Few-Shot Learnerscited by 32,419
  5. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  6. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  7. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  8. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  9. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  10. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  11. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  12. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,733
38 more
  1. 2016Densely Connected Convolutional Networkscited by 6,720
  2. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
  3. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
  4. 2017Neural Discrete Representation Learningcited by 4,712
  5. 2020Training data-efficient image transformers & distillation through attentioncited by 4,586
  6. 2021Zero-Shot Text-to-Image Generationcited by 4,288
  7. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,864
  8. 2014Deep Learning Face Attributes in the Wildcited by 3,693
  9. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,857
  10. 2016Aggregated Residual Transformations for Deep Neural Networkscited by 2,546
  11. 2016Pyramid Scene Parsing Networkcited by 2,479
  12. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,375
  13. 2018Deep contextualized word representationscited by 2,290
  14. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,287
  15. 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,222
  16. 2020FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidencecited by 2,085
  17. 2021Align before Fuse: Vision and Language Representation Learning with Momentum Distillationcited by 1,619
  18. 2018Unsupervised Representation Learning by Predicting Image Rotationscited by 1,591
  19. 2021LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairscited by 1,555
  20. 2019LXMERT: Learning Cross-Modality Encoder Representations from Transformerscited by 1,553
  21. 2016Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzlescited by 1,382
  22. 2020Big Self-Supervised Models are Strong Semi-Supervised Learnerscited by 1,380
  23. 2019Contrastive Multiview Codingcited by 1,301
  24. 2018Unified Perceptual Parsing for Scene Understandingcited by 1,271
  25. 2020Oscar: Object-Semantics Aligned Pre-training for Vision-Language Taskscited by 1,214
  26. 2021StyleCLIP: Text-Driven Manipulation of StyleGAN Imagerycited by 991
  27. 2019VL-BERT: Pre-training of Generic Visual-Linguistic Representationscited by 954
  28. 2017Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learningcited by 920
  29. 2017Revisiting Unreasonable Effectiveness of Data in Deep Learning Eracited by 919
  30. 2019SpanBERT: Improving Pre-training by Representing and Predicting Spanscited by 834
  31. 2019Big Transfer (BiT): General Visual Representation Learningcited by 769
  32. 2019Learning Representations by Maximizing Mutual Information Across Viewscited by 734
  33. 2019MaskGAN: Towards Diverse and Interactive Facial Image Manipulationcited by 628
  34. 2016MS-Celeb-1M: A Dataset and Benchmark for Large-Scale Face Recognitioncited by 624
  35. 2018Exploring the Limits of Weakly Supervised Pretrainingcited by 605
  36. 2019Unicoder-VL: A Universal Encoder for Vision and Language by Cross-modal Pre-trainingcited by 451
  37. 2019Using Pre-Training Can Improve Model Robustness and Uncertaintycited by 373
  38. 2014Discriminative Unsupervised Feature Learning with Exemplar Convolutional Neural Networkscited by 300

50 most cited of 75 All 75 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 15; 2023: 28; 2024: 62; 2025: 55; 2026: 35.

195 papers.

Sort:

By area:

  1. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,152
  2. 2024InstantID: Zero-shot Identity-Preserving Generation in Secondscited by 436
  3. 2023PhotoMaker: Customizing Realistic Human Photos via Stacked ID Embeddingcited by 328
  4. 2022MaskCLIP: Masked Self-Distillation Advances Contrastive Language-Image Pretrainingcited by 187
  5. 2024ID-Animator: Zero-Shot Identity-Preserving Human Video Generationcited by 124
  6. 2025EasyControl: Adding Efficient and Flexible Control for Diffusion Transformercited by 119
  7. 2024Arc2Face: A Foundation Model for ID-Consistent Human Facescited by 101
  8. 2022Masked Video Distillation: Rethinking Masked Feature Modeling for Self-supervised Video Representation Learningcited by 90
  9. 2024MovieDreamer: Hierarchical Generation for Coherent Long Visual Sequencecited by 89
  10. 2025Unified Multimodal Understanding and Generation Models: Advances, Challenges, and Opportunitiescited by 85
  11. 2022Learning Neural Parametric Head Modelscited by 73
  12. 2022Membership Inference Attacks Against Text-to-image Generation Modelscited by 69
38 more
  1. 2022Bootstrapped Masked Autoencoders for Vision BERT Pretrainingcited by 60
  2. 2022MARLIN: Masked Autoencoder for facial video Representation LearnINgcited by 57
  3. 2023HeadSculpt: Crafting 3D Head Avatars with Textcited by 49
  4. 2023Prompting Visual-Language Models for Dynamic Facial Expression Recognitioncited by 44
  5. 2023Attribute-preserving Face Dataset Anonymization via Latent Code Optimizationcited by 43
  6. 2023STAR Loss: Reducing Semantic Ambiguity in Facial Landmark Detectioncited by 43
  7. 2024ConsistentID: Portrait Generation with Multimodal Fine-Grained Identity Preservingcited by 43
  8. 2023DiffusionAvatars: Deferred Diffusion for High-fidelity 3D Head Avatarscited by 42
  9. 2024Infinite-ID: Identity-preserved Personalization via ID-semantics Decoupling Paradigmcited by 39
  10. 2023MonoNPHM: Dynamic Head Reconstruction from Monocular Videoscited by 35
  11. 2024FaceXFormer: A Unified Transformer for Facial Analysiscited by 34
  12. 2024Attacks and Defenses for Generative Diffusion Models: A Comprehensive Surveycited by 32
  13. 2023EmoCLIP: A Vision-Language Method for Zero-Shot Video Facial Expression Recognitioncited by 31
  14. 2024FlashFace: Human Image Personalization with High-fidelity Identity Preservationcited by 31
  15. 2024A Survey on Personalized Content Synthesis with Diffusion Modelscited by 29
  16. 2023Recurrence without Recurrence: Stable Video Landmark Detection with Deep Equilibrium Modelscited by 27
  17. 2024Identifying Implicit Social Biases in Vision-Language Modelscited by 27
  18. 2025FaceXBench: Evaluating Multimodal LLMs on Face Understandingcited by 24
  19. 2024FSFM: A Generalizable Face Security Foundation Model via Self-Supervised Facial Representation Learningcited by 23
  20. 2023HeadArtist: Text-conditioned 3D Head Generation with Self Score Distillationcited by 22
  21. 2022High-Res Facial Appearance Capture from Polarized Smartphone Imagescited by 21
  22. 2025Pixel3DMM: Versatile Screen-Space Priors for Single-Image 3D Face Reconstructioncited by 20
  23. 2023A Call to Reflect on Evaluation Practices for Age Estimation: Comparative Analysis of the State-of-the-Art and a Unified Benchmarkcited by 19
  24. 2025HumanAesExpert: Advancing a Multi-Modality Foundation Model for Human Image Aesthetic Assessmentcited by 19
  25. 2023Steered Diffusion: A Generalized Framework for Plug-and-Play Conditional Image Synthesiscited by 18
  26. 2023SVFAP: Self-supervised Video Facial Affect Perceivercited by 18
  27. 2024Enhancing Zero-Shot Facial Expression Recognition by LLM Knowledge Transfercited by 18
  28. 2024Face Anonymization Made Simplecited by 18
  29. 2024Omni-ID: Holistic Identity Representation Designed for Generative Taskscited by 18
  30. 2022Pose-disentangled Contrastive Learning for Self-supervised Facial Representationcited by 17
  31. 2024Self-Supervised Facial Representation Learning with Facial Region Awarenesscited by 17
  32. 2025DiverseFlow: Sample-Efficient Diverse Mode Coverage in Flowscited by 17
  33. 2023Weakly-Supervised Text-driven Contrastive Learning for Facial Behavior Understandingcited by 16
  34. 2024ExpLLM: Towards Chain of Thought for Facial Expression Recognitioncited by 15
  35. 2023Implicit Neural Head Synthesis via Controllable Local Deformation Fieldscited by 14
  36. 2023A Generalist FaceX via Learning Unified Facial Representationcited by 14
  37. 2024Deformable One-shot Face Stylization via DINO Semantic Guidancecited by 14
  38. 2024DiffusionAct: Controllable Diffusion Autoencoder for One-shot Face Reenactmentcited by 14

50 most cited of 195 All 195 (JSON)