mlsanityData to 7 Oct 2026

Subjective-Aligned Dataset and Metric for Text-to-Video Quality Assessment

arXiv 2403.11956 ↗First posted 18 Mar 2024cs.CVcited by 39

Not on the map. Why?

Abstract (arXiv)

With the rapid development of generative models, Artificial Intelligence-Generated Contents (AIGC) have exponentially increased in daily lives. Among them, Text-to-Video (T2V) generation has received widespread attention. Though many T2V models have been released for generating high perceptual quality videos, there is still lack of a method to evaluate the quality of these videos quantitatively. To solve this issue, we establish the largest-scale Text-to-Video Quality Assessment DataBase (T2VQA-DB) to date. The dataset is composed of 10,000 videos generated by 9 different T2V models. We also conduct a subjective study to obtain each video's corresponding mean opinion score. Based on T2VQA-DB, we propose a novel transformer-based model for subjective-aligned Text-to-Video Quality Assessment (T2VQA). The model extracts features from text-video alignment and video fidelity perspectives, then it leverages the ability of a large language model to give the prediction score. Experimental results show that T2VQA outperforms existing T2V metrics and SOTA video quality assessment models. Quantitative analysis indicates that T2VQA is capable of giving subjective-align predictions, validating its effectiveness. The dataset and code will be released at https://github.com/QMME/T2VQA.

Key facts. Cited by 39 papers in the map (data through 7 Oct 2026). Builds on 47 of its 69 references that are in the map. First posted on arXiv 18 Mar 2024.

Builds on

47 of 69 references.

References · 47

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  3. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  4. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  5. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  6. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  7. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
  8. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,259
  9. 2016Improved Techniques for Training GANscited by 2,938
  10. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,855
  11. 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,755
  12. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,725
35 more
  1. 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuningcited by 1,418
  2. 2018Towards Accurate Generative Models of Video: A New Metric & Challengescited by 1,204
  3. 2021Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrievalcited by 1,175
  4. 2023ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generationcited by 1,010
  5. 2022CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformerscited by 973
  6. 2023Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generationcited by 950
  7. 2023Text2Video-Zero: Text-to-Image Diffusion Models are Zero-Shot Video Generatorscited by 666
  8. 2023ModelScope Text-to-Video Technical Reportcited by 651
  9. 2017Can Spatiotemporal 3D CNNs Retrace the History of 2D CNNs and ImageNet?cited by 551
  10. 2023mPLUG-Owl2: Revolutionizing Multi-modal Large Language Model with Modality Collaborationcited by 545
  11. 2023InternVid: A Large-scale Video-Text Dataset for Multimodal Understanding and Generationcited by 488
  12. 2022MagicVideo: Efficient Video Generation With Latent Diffusion Modelscited by 463
  13. 2023EvalCrafter: Benchmarking and Evaluating Large Video Generation Modelscited by 332
  14. 2023VideoFusion: Decomposed Diffusion Models for High-Quality Video Generationcited by 330
  15. 2022Exploring Video Quality Assessment on User Generated Contents from Aesthetic and Technical Perspectivescited by 314
  16. 2022CogView2: Faster and Better Text-to-Image Generation via Hierarchical Transformerscited by 305
  17. 2023Human Preference Score: Better Aligning Text-to-Image Models with Human Preferencecited by 284
  18. 2021GODIVA: Generating Open-DomaIn Videos from nAtural Descriptionscited by 254
  19. 2023Q-Bench: A Benchmark for General-Purpose Foundation Models on Low-level Visioncited by 247
  20. 2022Selective Annotation Makes Language Models Better Few-Shot Learnerscited by 242
  21. 2022FAST-VQA: Efficient End-to-end Video Quality Assessment with Fragment Samplingcited by 179
  22. 2023Q-Instruct: Improving Low-level Visual Abilities for Multi-modality Foundation Modelscited by 152
  23. 2022A Deep Learning based No-reference Quality Assessment Model for UGC Videoscited by 128
  24. 2023FETV: A Benchmark for Fine-Grained Evaluation of Open-Domain Text-to-Video Generationcited by 119
  25. 2024Towards Open-ended Visual Quality Comparisoncited by 85
  26. 2023A Perceptual Quality Assessment Exploration for AIGC Imagescited by 54
  27. 2024Towards A Better Metric for Text-to-Video Generationcited by 50
  28. 2023Reuse and Diffuse: Iterative Denoising for Text-to-Video Generationcited by 49
  29. 2023Advancing Zero-Shot Digital Human Quality Assessment through Text-Prompted Evaluationcited by 27
  30. 2023Exploring Opinion-unaware Video Quality Assessment with Semantic Affinity Criterioncited by 23
  31. 2023Measuring the Quality of Text-to-Video Model Outputs: Metrics and Datasetcited by 23
  32. 2023Q-Boost: On Visual Quality Assessment Ability of Low-level Multi-Modality Foundation Modelscited by 22
  33. 2024Q-Refine: A Perceptual Quality Refiner for AI-Generated Imagecited by 21
  34. 2023StableVQA: A Deep No-Reference Quality Assessment Model for Video Stabilitycited by 15
  35. 2023Light-VQA: A Multi-Dimensional Quality Assessment Model for Low-Light Video Enhancementcited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 14; 2025: 19; 2026: 6.

39 papers.

Sort:

By area:

  1. 2024LiFT: Leveraging Human Feedback for Text-to-Video Model Alignmentcited by 51
  2. 2024Diffusion Model-Based Video Editing: A Surveycited by 44
  3. 2024Q-Bench-Video: Benchmarking the Video Quality Understanding of LMMscited by 39
  4. 2025VideoScore2: Think before You Score in Generative Video Evaluationcited by 37
  5. 2024AIGIQA-20K: A Large Database for AI-Generated Image Quality Assessmentcited by 33
  6. 2024AIGV-Assessor: Benchmarking and Evaluating the Perceptual Quality of Text-to-Video Generation with LMMcited by 24
  7. 2025VideoGen-Eval: Agent-based System for Video Generation Evaluationcited by 23
  8. 2025FingER: Content Aware Fine-grained Evaluation with Reasoning for AI-Generated Videoscited by 22
  9. 2024Beyond FVD: Enhanced Evaluation Metrics for Video Generation Qualitycited by 21
  10. 2025Can World Simulators Reason? Gen-ViRe: A Generative Visual Reasoning Benchmarkcited by 19
  11. 2024Video Quality Assessment: A Comprehensive Surveycited by 15
  12. 2024Exploring AIGC Video Quality: A Focus on Visual Harmony, Video-Text Consistency and Domain Distribution Gapcited by 13
27 more
  1. 2025LOVE: Benchmarking and Evaluating Text-to-Video Generation and Video-to-Text Interpretationcited by 10
  2. 2025TDVE-Assessor: Benchmarking and Evaluating the Quality of Text-Driven Video Editing with LMMscited by 10
  3. 2025UI2V-Bench: An Understanding-based Image-to-video Generation Benchmarkcited by 10
  4. 2024Human-Activity AGV Quality Assessment: A Benchmark Dataset and an Objective Evaluation Metriccited by 9
  5. 2024Neuro-Symbolic Evaluation of Text-to-Video Models using Formal Verificationcited by 9
  6. 2025We'll Fix it in Post: Improving Text-to-Video Generation with Neuro-Symbolic Feedbackcited by 9
  7. 2025Aligning Anime Video Generation with Human Feedbackcited by 8
  8. 2026VQQA: An Agentic Approach for Video Evaluation and Quality Improvementcited by 8
  9. 2024Video Is Worth a Thousand Images: Exploring the Latest Trends in Long Video Generationcited by 7
  10. 2025GRADEO: Towards Human-Like Evaluation for Text-to-Video Generation via Multi-Step Reasoningcited by 7
  11. 2025LMME3DHF: Benchmarking and Evaluating Multimodal 3D Human Face Generation with LMMscited by 7
  12. 2024Light-VQA+: A Video Quality Assessment Model for Exposure Correction with Vision-Language Guidancecited by 6
  13. 2024CMC-Bench: Towards a New Paradigm of Visual Signal Compressioncited by 6
  14. 2025Perceptual Visual Quality Assessment: Principles, Methods, and Future Directionscited by 5
  15. 2025AGAV-Rater: Adapting Large Multimodal Model for AI-Generated Audio-Visual Quality Assessmentcited by 4
  16. 2024Quality Prediction of AI Generated Images and Videos: Emerging Trends and Opportunitiescited by 3
  17. 2026CaC: Advancing Video Reward Models via Hierarchical Spatiotemporal Concentratingcited by 3
  18. 2025Multilevel Semantic-Aware Model for AI-Generated Video Quality Assessmentcited by 2
  19. 2025Towards Holistic Visual Quality Assessment of AI-Generated Videos: A LLM-Based Multi-Dimensional Evaluation Modelcited by 2
  20. 2025Q-Router: Agentic Video Quality Assessment with Expert Model Routing and Artifact Localizationcited by 2
  21. 2026Thinking with Frames: Generative Video Distortion Evaluation via Frame Reward Modelcited by 2
  22. 2026Multi-Dimensional Quality Assessment for AI-Generated Human-Centric Videos: Dataset and Modelcited by 2
  23. 2025RAISE: Realness Assessment for Image Synthesis and Evaluationcited by 1
  24. 2026EduVQA: Towards Concept-Aware Assessment of Educational AI-Generated Videoscited by 1
  25. 2026EduStory: A Unified Framework for Pedagogically-Consistent Multi-Shot STEM Instructional Video Generationcited by 1
  26. 2025Evaluating Text-to-Image and Text-to-Video Synthesis with a Conditional Fr'echet Distancecited by 0
  27. 2025Better Supervised Fine-tuning for VQA: Integer-Only Losscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025MVQA-68K: A Multi-dimensional and Causally-annotated Dataset with Quality Interpretability for Video Assessment
  2. 2024PCQA: A Strong Baseline for AIGC Quality Assessment Based on Prompt Condition
  3. 2025Q-Save: Towards Scoring and Attribution for Generated Video Evaluation
  4. 2024VE-Bench: Subjective-Aligned Benchmark Suite for Text-Driven Video Editing Quality Assessment
  5. 2025Score2Instruct: Scaling Up Video Quality-Centric Instructions via Automated Dimension Scoring
  6. 2025CAMP-VQA: Caption-Embedded Multimodal Perception for No-Reference Quality Assessment of Compressed Video
  7. 2026DPC-VQA: Decoupling Quality Perception and Residual Calibration for Video Quality Assessment
  8. 2025VQ-Insight: Teaching VLMs for AI-Generated Video Quality Understanding via Progressive Visual Reinforcement Learning