mlsanityData to 14 Oct 2026

FIFA: Unified Faithfulness Evaluation Framework for Text-to-Video and Video-to-Text Generation

arXiv 2507.06523 ↗First posted 9 Jul 2025cs.CV cs.CL cs.GRcited by 1

On the map: Multimodal LLMs / vision-language models → Video understanding with MLLMs

Abstract (arXiv)

Video Multimodal Large Language Models (VideoMLLMs) have achieved remarkable progress in both Video-to-Text and Text-to-Video tasks. However, they often suffer fro hallucinations, generating content that contradicts the visual input. Existing evaluation methods are limited to one task (e.g., V2T) and also fail to assess hallucinations in open-ended, free-form responses. To address this gap, we propose FIFA, a unified FaIthFulness evAluation framework that extracts comprehensive descriptive facts, models their semantic dependencies via a Spatio-Temporal Semantic Dependency Graph, and verifies them using VideoQA models. We further introduce Post-Correction, a tool-based correction framework that revises hallucinated content. Extensive experiments demonstrate that FIFA aligns more closely with human judgment than existing evaluation methods, and that Post-Correction effectively improves factual consistency in both text and video generation.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Video understanding with MLLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 29 of its 104 references that are in the map. First posted on arXiv 9 Jul 2025.

Builds on

29 of 104 references.

Closest ancestors · 2

Same direction, newest first

  1. 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projection16 Nov
  2. 2023Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understanding5 Jun

Other references · 27

  1. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
  2. 2017Neural Discrete Representation Learningcited by 4,712
  3. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,424
  4. 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,261
  5. 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,660
  6. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,599
  7. 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,282
  8. 2021VideoGPT: Video Generation using VQ-VAE and Transformerscited by 658
  9. 2024VideoCrafter2: Overcoming Data Limitations for High-Quality Video Diffusion Modelscited by 579
  10. 2025VideoLLaMA 3: Frontier Multimodal Foundation Models for Image and Video Understandingcited by 518
  11. 2022MagicVideo: Efficient Video Generation With Latent Diffusion Modelscited by 465
  12. 2023Mitigating Hallucination in Large Multi-Modal Models via Robust Instruction Tuningcited by 445
15 more
  1. 2023TIFA: Accurate and Interpretable Text-to-Image Faithfulness Evaluation with Question Answeringcited by 428
  2. 2023EvalCrafter: Benchmarking and Evaluating Large Video Generation Modelscited by 340
  3. 2022Expanding Language-Image Pretrained Models for General Video Recognitioncited by 321
  4. 2025VBench-2.0: Advancing Video Generation Benchmark Suite for Intrinsic Faithfulnesscited by 315
  5. 2021N"UWA: Visual Synthesis Pre-training for Neural visUal World creAtioncited by 253
  6. 2022FAST-VQA: Efficient End-to-end Video Quality Assessment with Fragment Samplingcited by 179
  7. 2023Make Pixels Dance: High-Dynamic Video Generationcited by 140
  8. 2023Volcano: Mitigating Multimodal Hallucination through Self-Feedback Guided Revisioncited by 98
  9. 2024VideoHallucer: Evaluating Intrinsic and Extrinsic Hallucinations in Large Video-Language Modelscited by 90
  10. 2018To Create What You Tell: Generating Videos from Captionscited by 84
  11. 2023Vista-LLaMA: Reducing Hallucination in Video Language Models via Equal Distance to Visual Tokenscited by 71
  12. 2024VidHalluc: Evaluating Temporal Hallucinations in Multimodal Large Language Models for Video Understandingcited by 68
  13. 2024EventHallusion: Diagnosing Event Hallucinations in Video LLMscited by 57
  14. 2024Towards A Better Metric for Text-to-Video Generationcited by 51
  15. 2022Thinking Hallucination for Video Captioningcited by 5

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025LoCoT2V-Bench: Benchmarking Long-Form and Complex Text-to-Video Generationcited by 7

Similar, not cited

Close in wording, no citation either way

  1. 2026MLLM-Guided Semantic Correction for Text-to-Video Generation
  2. 2025Video SimpleQA: Towards Factuality Evaluation in Large Video Language Models
  3. 2025T2VWorldBench: A Benchmark for Evaluating World Knowledge in Text-to-Video Generation
  4. 2025ResNetVLLM-2: Addressing ResNetVLLM's Multi-Modal Hallucinations
  5. 2025Taming Hallucinations: Boosting MLLMs' Video Understanding via Counterfactual Video Generation
  6. 2026DualFact+: A Multimodal Fact Verification Framework for Procedural Video Understanding
  7. 2025VideoScore2: Think before You Score in Generative Video Evaluation
  8. 2026Meta-Video: Efficient Standardized Evaluation for Video Understanding