mlsanityData to 14 Oct 2026

F-Bench: Rethinking Human Preference Evaluation Metrics for Benchmarking Face Generation, Customization, and Restoration

arXiv 2412.13155 ↗First posted 17 Dec 2024cs.CVcited by 10

Not on the map. Why?

Abstract (arXiv)

Artificial intelligence generative models exhibit remarkable capabilities in content creation, particularly in face image generation, customization, and restoration. However, current AI-generated faces (AIGFs) often fall short of human preferences due to unique distortions, unrealistic details, and unexpected identity shifts, underscoring the need for a comprehensive quality evaluation framework for AIGFs. To address this need, we introduce FaceQ, a large-scale, comprehensive database of AI-generated Face images with fine-grained Quality annotations reflecting human preferences. The FaceQ database comprises 12,255 images generated by 29 models across three tasks: (1) face generation, (2) face customization, and (3) face restoration. It includes 32,742 mean opinion scores (MOSs) from 180 annotators, assessed across multiple dimensions: quality, authenticity, identity (ID) fidelity, and text-image correspondence. Using the FaceQ database, we establish F-Bench, a benchmark for comparing and evaluating face generation, customization, and restoration models, highlighting strengths and weaknesses across various prompts and evaluation dimensions. Additionally, we assess the performance of existing image quality assessment (IQA), face quality assessment (FQA), AI-generated content image quality assessment (AIGCIQA), and preference evaluation metrics, manifesting that these standard metrics are relatively ineffective in evaluating authenticity, ID fidelity, and text-image correspondence. The FaceQ database will be publicly available upon publication.

Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Builds on 70 of its 114 references that are in the map. First posted on arXiv 17 Dec 2024.

Builds on

70 of 114 references.

References · 70

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  5. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  6. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  7. 2020Denoising Diffusion Implicit Modelscited by 9,781
  8. 2023Visual Instruction Tuningcited by 7,869
  9. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  10. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  11. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
  12. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
38 more
  1. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,322
  2. 2021Zero-Shot Text-to-Image Generationcited by 4,288
  3. 2014Deep Learning Face Attributes in the Wildcited by 3,693
  4. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,300
  5. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,598
  6. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,424
  7. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,245
  8. 2023IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Modelscited by 1,464
  9. 2022Imagic: Text-Based Real Image Editing with Diffusion Modelscited by 1,178
  10. 2022Multi-Concept Customization of Text-to-Image Diffusioncited by 1,106
  11. 2023Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generationcited by 977
  12. 2021Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Datacited by 970
  13. 2023PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesiscited by 864
  14. 2022MANIQA: Multi-dimension Attention Network for No-Reference Image Quality Assessmentcited by 522
  15. 2023Exploiting Diffusion Prior for Real-World Image Super-Resolutioncited by 489
  16. 2024InstantID: Zero-shot Identity-Preserving Generation in Secondscited by 436
  17. 2023ELITE: Encoding Visual Concepts into Textual Embeddings for Customized Text-to-Image Generationcited by 416
  18. 2021Towards Real-World Blind Face Restoration with Generative Facial Priorcited by 349
  19. 2023PhotoMaker: Customizing Realistic Human Photos via Stacked ID Embeddingcited by 328
  20. 2023FastComposer: Tuning-Free Multi-Subject Image Generation with Localized Attentioncited by 313
  21. 2015WIDER FACE: A Face Detection Benchmarkcited by 298
  22. 2024Hunyuan-DiT: A Powerful Multi-Resolution Diffusion Transformer with Fine-Grained Chinese Understandingcited by 298
  23. 2023Human Preference Score: Better Aligning Text-to-Image Models with Human Preferencecited by 287
  24. 2023DiffBIR: Towards Blind Image Restoration with Generative Diffusion Priorcited by 257
  25. 2024Playground v2.5: Three Insights towards Enhancing Aesthetic Quality in Text-to-Image Generationcited by 256
  26. 2023DiffIR: Efficient Diffusion Model for Image Restorationcited by 255
  27. 2022Towards Robust Blind Face Restoration with Codebook Lookup Transformercited by 231
  28. 2015The MegaFace Benchmark: 1 Million Faces for Recognition at Scalecited by 200
  29. 2021GAN Prior Embedded Network for Blind Face Restoration in the Wildcited by 197
  30. 2022VFHQ: A High-Quality Dataset and Benchmark for Video Face Super-Resolutioncited by 176
  31. 2017FSRNet: End-to-End Learning Face Super-Resolution with Facial Priorscited by 170
  32. 2021No-Reference Image Quality Assessment via Transformers, Relative Ranking, and Self-Consistencycited by 166
  33. 2023AGIQA-3K: An Open Database for AI-Generated Image Quality Assessmentcited by 142
  34. 2023Rich Human Feedback for Text-to-Image Generationcited by 141
  35. 2024ID-Animator: Zero-Shot Identity-Preserving Human Video Generationcited by 124
  36. 2022VQFR: Blind Face Restoration with Vector-Quantized Dictionary and Parallel Decodercited by 116
  37. 2020SER-FIQ: Unsupervised Estimation of Face Image Quality Based on Stochastic Embedding Robustnesscited by 112
  38. 2020Progressive Semantic-Aware Style Transformation for Blind Face Restorationcited by 99

50 most cited of 70 All 70 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 7.

10 papers.

  1. 2025AGHI-QA: A Subjective-Aligned Dataset and Metric for AI-Generated Human Imagescited by 13
  2. 2025MoA-VR: A Mixture-of-Agents System Towards All-in-One Video Restorationcited by 12
  3. 2026Agentic Retoucher for Text-To-Image Generationcited by 7
  4. 2026ConsistentRFT: Reducing Visual Hallucinations in Flow-based Reinforcement Fine-Tuningcited by 6
  5. 2026Evaluating Image Editing with LLMs: A Comprehensive Benchmark and Intermediate-Layer Probing Approachcited by 3
  6. 2026Market-Bench: Benchmarking Large Language Models on Economic and Trade Competitioncited by 3
  7. 2025Quality Assessment and Distortion-aware Saliency Prediction for AI-Generated Omnidirectional Imagescited by 2
  8. 2026Multi-Dimensional Quality Assessment for AI-Generated Human-Centric Videos: Dataset and Modelcited by 2
  9. 2026PixelSmile: Toward Fine-Grained Facial Expression Editingcited by 1
  10. 2026A2BFR: Attribute-Aware Blind Face Restorationcited by 0