mlsanityData to 7 Oct 2026

15M Multimodal Facial Image-Text Dataset

arXiv 2407.08515 ↗First posted 11 Jul 2024cs.CV cs.AIcited by 14

Not on the map. Why?

Abstract (arXiv)

Currently, image-text-driven multi-modal deep learning models have demonstrated their outstanding potential in many fields. In practice, tasks centered around facial images have broad application prospects. This paper presents FaceCaption-15M, a large-scale, diverse, and high-quality dataset of facial images accompanied by their natural language descriptions (facial image-to-text). This dataset aims to facilitate a study on face-centered tasks. FaceCaption-15M comprises over 15 million pairs of facial images and their corresponding natural language descriptions of facial features, making it the largest facial image-caption dataset to date. We conducted a comprehensive analysis of image quality, text naturalness, text complexity, and text-image relevance to demonstrate the superiority of FaceCaption-15M. To validate the effectiveness of FaceCaption-15M, we first trained a facial language-image pre-training model (FLIP, similar to CLIP) to align facial image with its corresponding captions in feature space. Subsequently, using both image and text encoders and fine-tuning only the linear layer, our FLIP-based models achieved state-of-the-art results on two challenging face-centered tasks. The purpose is to promote research in the field of face-related tasks through the availability of the proposed FaceCaption-15M dataset. All data, codes, and models are publicly available. https://huggingface.co/datasets/OpenFace-CQUPT/FaceCaption-15M

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Builds on 66 of its 87 references that are in the map. First posted on arXiv 11 Jul 2024.

Builds on

66 of 87 references.

References · 66

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  5. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  6. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  7. 2022Training language models to follow instructions with human feedbackcited by 15,712
  8. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  9. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  10. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  11. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  12. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
38 more
  1. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  2. 2014Deep Learning Face Attributes in the Wildcited by 3,679
  3. 2023Qwen Technical Reportcited by 3,273
  4. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  5. 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,967
  6. 2015FaceNet: A Unified Embedding for Face Recognition and Clusteringcited by 2,833
  7. 2015VQA: Visual Question Answeringcited by 2,489
  8. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,360
  9. 2020FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidencecited by 2,080
  10. 2021Align before Fuse: Vision and Language Representation Learning with Momentum Distillationcited by 1,612
  11. 2018Unsupervised Representation Learning by Predicting Image Rotationscited by 1,586
  12. 2021LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairscited by 1,544
  13. 2016Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzlescited by 1,379
  14. 2021GLM: General Language Model Pretraining with Autoregressive Blank Infillingcited by 1,026
  15. 2018Deep Clustering for Unsupervised Learning of Visual Featurescited by 1,020
  16. 2022Exploring CLIP for Assessing the Look and Feel of Imagescited by 974
  17. 2017Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learningcited by 918
  18. 2017VGGFace2: A dataset for recognising faces across pose and agecited by 911
  19. 2020Encoding in Style: a StyleGAN Encoder for Image-to-Image Translationcited by 727
  20. 2019Interpreting the Latent Space of GANs for Semantic Face Editingcited by 691
  21. 2023Baichuan 2: Open Large-scale Language Modelscited by 664
  22. 2019MaskGAN: Towards Diverse and Interactive Facial Image Manipulationcited by 624
  23. 2014Learning Face Representation from Scratchcited by 451
  24. 2019VATEX: A Large-Scale, High-Quality Multilingual Dataset for Video-and-Language Researchcited by 412
  25. 2022AdaFace: Quality Adaptive Margin for Face Recognitioncited by 390
  26. 2022CelebV-HQ: A Large-Scale Video Facial Attributes Datasetcited by 246
  27. 2015The MegaFace Benchmark: 1 Million Faces for Recognition at Scalecited by 199
  28. 2021General Facial Representation Learning in a Visual-Linguistic Mannercited by 193
  29. 2021ElasticFace: Elastic Margin Loss for Deep Face Recognitioncited by 143
  30. 2023Collaborative Diffusion for Multi-Modal Face Generation and Editingcited by 116
  31. 2021FENeRF: Face Editing in Neural Radiance Fieldscited by 112
  32. 2019SC-FEGAN: Face Editing Generative Adversarial Network with User's Sketch and Colorcited by 105
  33. 2023CelebV-Text: A Large-Scale Facial Text-Video Datasetcited by 103
  34. 2021Talk-to-Edit: Fine-Grained Facial Editing via Dialogcited by 102
  35. 2023Vita-CLIP: Video and text adaptive CLIP via Multimodal Promptingcited by 90
  36. 2020Sketch Less for More: On-the-Fly Fine-Grained Sketch Based Image Retrievalcited by 59
  37. 2023PoseFix: Correcting 3D Human Poses with Natural Languagecited by 38
  38. 2022Sketch3T: Test-Time Training for Zero-Shot SBIRcited by 34

50 most cited of 66 All 66 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 5; 2026: 6.

14 papers.

Sort:

By area:

  1. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,573
  2. 2025FaceXBench: Evaluating Multimodal LLMs on Face Understandingcited by 23
  3. 2026InternVL-U: Democratizing Unified Multimodal Models for Understanding, Reasoning, Generation and Editingcited by 23
  4. 2025FaceBench: A Multi-View Multi-Level Facial Attribute VQA Dataset for Benchmarking Face Perception MLLMscited by 11
  5. 2024HumanVLM: Foundation for Human-Scene Vision-Language Modelcited by 8
  6. 2026Condition Matters in Full-head 3D GANscited by 2
  7. 2024CA-Edit: Causality-Aware Condition Adapter for High-Fidelity Local Facial Attribute Editingcited by 1
  8. 2026MeFEm: Medical Face Embedding modelcited by 1
  9. 2026DataComp-VLM: Improved Open Datasets for Vision-Language Modelscited by 1
  10. 2025OCC-MLLM-CoT-Alpha: Towards Multi-stage Occlusion Recognition Based on Large Language Models via 3D-Aware Supervision and Chain-of-Thoughts Guidancecited by 0
  11. 2025Learning Transferable Facial Emotion Representations from Large-Scale Semantically Rich Captionscited by 0
  12. 2025Face-MakeUpV2: Facial Consistency Learning for Controllable Text-to-Image Generationcited by 0
2 more
  1. 2026Square Superpixel Generation and Representation Learning via Granular Ball Computingcited by 0
  2. 2026Unmasking Face Embeddings: Reading, Rendering and Naming with Foundation Modelscited by 0