mlsanityData to 14 Oct 2026

VCIFBench: Evaluating Complex Instruction Following for Video Understanding

arXiv 2606.04588 ↗First posted 3 Jun 2026cs.CLcited by 0

On the map: Multimodal LLMs / vision-language models → Video understanding with MLLMs

Abstract (arXiv)

Multimodal large language models have made rapid progress in video understanding, yet existing benchmarks largely rely on simple prompts and provide limited evidence about whether models can satisfy explicit output constraints. We introduce VCIFBench, a benchmark for evaluating complex instruction following in video understanding. VCIFBench constructs constraint-rich instructions from both benchmark-adapted and directly video-grounded prompts, covering content, format, style, and structure requirements, and evaluates model outputs with a hybrid verification pipeline. The benchmark contains 306 satisfiable test instructions, 540 DPO training instances, and a 100-item diagnostic set for evaluating whether models can recognize instruction conflicts. Experiments on 10 MLLMs show that joint constraint satisfaction remains challenging. Preference optimization improves instruction following for two model families, while Conflict-100 reveals that models usually execute a satisfiable-looking subset instead of detecting global incompatibility.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Video understanding with MLLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 21 of its 33 references that are in the map. First posted on arXiv 3 Jun 2026.

Builds on

21 of 33 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis31 May

Other references · 20

  1. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  2. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
  3. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,876
  4. 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,141
  5. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,080
  6. 2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilitiescited by 1,239
  7. 2021NExT-QA:Next Phase of Question-Answering to Explaining Temporal Actionscited by 744
  8. 2017Towards Automatic Learning of Procedures from Web Instructional Videoscited by 619
  9. 2023AppAgent: Multimodal Agents as Smartphone Userscited by 415
  10. 2024TempCompass: Do Video LLMs Really Understand Videos?cited by 363
  11. 2024MLVU: Benchmarking Multi-task Long Video Understandingcited by 292
  12. 2024Benchmarking Complex Instruction-Following with Multiple Constraints Compositioncited by 152
8 more
  1. 2023Controlled Text Generation with Natural Language Instructionscited by 87
  2. 2024MME-Survey: A Comprehensive Survey on Evaluation of Multimodal LLMscited by 72
  3. 2023Can Large Language Models Understand Real-World Complex Instructions?cited by 69
  4. 2024CFBench: A Comprehensive Constraints-Following Benchmark for LLMscited by 67
  5. 2025ComplexFuncBench: Exploring Multi-Step and Constrained Function Calling under Long-Context Scenariocited by 55
  6. 2024MMWorld: Towards Multi-discipline Multi-faceted World Model Evaluation in Videoscited by 43
  7. 2025MM-IFEngine: Towards Multimodal Instruction Followingcited by 42
  8. 2026QEVA: A Reference-Free Evaluation Metric for Narrative Video Summarization with Multimodal Question Answeringcited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Video-IFBench: Evaluating Instruction Following of Multimodal LLMs in Video Understanding Scenarios
  2. 2025InstructionBench: An Instructional Video Understanding Benchmark
  3. 2026Empowering Reliable Visual-Centric Instruction Following in MLLMs
  4. 2024MMBench-Video: A Long-Form Multi-Shot Benchmark for Holistic Video Understanding
  5. 2025VideoRewardBench: Comprehensive Evaluation of Multimodal Reward Models for Video Understanding
  6. 2026Meta-Video: Efficient Standardized Evaluation for Video Understanding
  7. 2023MVBench: A Comprehensive Multi-modal Video Understanding Benchmark
  8. 2026VirtueBench: Evaluating Trustworthiness under Uncertainty in Long Video Understanding