# MTAVG-Bench 2.0: Diagnosing Failure Modes of Cinematic Expressiveness in Multi-Talker Audio-Video Generation arXiv: https://arxiv.org/abs/2605.28035 ยท HTML: https://mlsanity.com/paper/2605.28035/ ## Key facts - Cited by 4 papers in the map (data through 14 Oct 2026). - Builds on 32 of its 41 references that are in the map. - First posted on arXiv 27 May 2026. ## Builds on (most cited first, 30 of 32) - [MiniCPM-V: A GPT-4V Level MLLM on Your Phone](https://mlsanity.com/paper/2408.01800/) (2024) - [VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMs](https://mlsanity.com/paper/2406.07476/) (2024) - [Seedance 1.0: Exploring the Boundaries of Video Generation Models](https://mlsanity.com/paper/2506.09113/) (2025) - [LTX-2: Efficient Joint Audio-Visual Foundation Model](https://mlsanity.com/paper/2601.03233/) (2026) - [VideoDirectorGPT: Consistent Multi-scene Video Generation via LLM-Guided Planning](https://mlsanity.com/paper/2309.15091/) (2023) - [Long Context Tuning for Video Generation](https://mlsanity.com/paper/2503.10589/) (2025) - [JavisDiT: Joint Audio-Video Diffusion Transformer with Hierarchical Spatio-Temporal Prior Synchronization](https://mlsanity.com/paper/2503.23377/) (2025) - [Ovi: Twin Backbone Cross-Modal Fusion for Audio-Video Generation](https://mlsanity.com/paper/2510.01284/) (2025) - [Ming-Omni: A Unified Multimodal Model for Perception and Generation](https://mlsanity.com/paper/2506.09344/) (2025) - [OmniVinci: Enhancing Architecture and Data for Omni-Modal Understanding LLM](https://mlsanity.com/paper/2510.15870/) (2025) - [Video-Bench: Human-Aligned Video Generation Benchmark](https://mlsanity.com/paper/2504.04907/) (2025) - [MOVA: Towards Scalable and Synchronized Video-Audio Generation](https://mlsanity.com/paper/2602.08794/) (2026) - [MovieFactory: Automatic Movie Creation from Text using Large Generative Models for Language and Images](https://mlsanity.com/paper/2306.07257/) (2023) - [Evaluation Agent: Efficient and Promptable Evaluation Framework for Visual Generative Models](https://mlsanity.com/paper/2412.09645/) (2024) - [UniAVGen: Unified Audio and Video Generation with Asymmetric Cross-Modal Interactions](https://mlsanity.com/paper/2511.03334/) (2025) - [MovieBench: A Hierarchical Movie Level Dataset for Long Video Generation](https://mlsanity.com/paper/2411.15262/) (2024) - [TAVGBench: Benchmarking Text to Audible-Video Generation](https://mlsanity.com/paper/2404.14381/) (2024) - [VideoHallu: Evaluating and Mitigating Multi-modal Hallucinations on Synthetic Video Understanding](https://mlsanity.com/paper/2505.01481/) (2025) - [FilMaster: Bridging Cinematic Principles and Generative AI for Automated Film Generation](https://mlsanity.com/paper/2506.18899/) (2025) - [JavisDiT++: Unified Modeling and Optimization for Joint Audio-Video Generation](https://mlsanity.com/paper/2602.19163/) (2026) - [T2AV-Compass: Towards Unified Evaluation for Text-to-Audio-Video Generation](https://mlsanity.com/paper/2512.21094/) (2025) - [STAGE: Storyboard-Anchored Generation for Cinematic Multi-shot Narrative](https://mlsanity.com/paper/2512.12372/) (2025) - [Dynamic Storyboard Generation in an Engine-based Virtual Environment for Video Production](https://mlsanity.com/paper/2301.12688/) (2023) - [Harmony: Harmonizing Audio and Video Generation through Cross-Task Synergy](https://mlsanity.com/paper/2511.21579/) (2025) - [AVGen-Bench: A Task-Driven Benchmark for Multi-Granular Evaluation of Text-to-Audio-Video Generation](https://mlsanity.com/paper/2604.08540/) (2026) - [DreamID-Omni: Unified Framework for Controllable Human-Centric Audio-Video Generation](https://mlsanity.com/paper/2602.12160/) (2026) - [Cinematic Behavior Transfer via NeRF-based Differentiable Filming](https://mlsanity.com/paper/2311.17754/) (2023) - [PhyAVBench: A Challenging Audio Physics-Sensitivity Benchmark for Physically Grounded Text-to-Audio-Video Generation](https://mlsanity.com/paper/2512.23994/) (2025) - [MTAVG-Bench: A Diagnostic Benchmark for Multi-Talker Dialogue-Centric Audio-Video Generation](https://mlsanity.com/paper/2602.00607/) (2026) - [The Script is All You Need: An Agentic Framework for Long-Horizon Dialogue-to-Cinematic Video Generation](https://mlsanity.com/paper/2601.17737/) (2026) ## Built on by (most cited first, 4 of 4) - [LightAIR: Lightweight Action Inversion and Riemannian Rectification for Text-based Person Anomaly Search](https://mlsanity.com/paper/2608.09152/) (2026) - [Motion-Aware Reasoning from Speech to Mask Tracks: Runner-up Solution for the MeViS-Audio Track of the 8th LSVOS Challenge 2026](https://mlsanity.com/paper/2608.22337/) (2026) - [DramaChain Bench: An End-to-End Benchmark for Short-Drama Generation](https://mlsanity.com/paper/2609.00646/) (2026) - [REMEDY: How Far Is Video Generation from Medical Education World Models?](https://mlsanity.com/paper/2609.32460/) (2026) ## Abstract (arXiv) In recent years, Multi-Talker Audio-Video Generation (MTAVG) models have shown promising performance on fundamental metrics such as lip-sync and audio-visual alignment. However, these metrics remain insufficient for assessing cinematic expressiveness in scene-level generation. In multi-character scenes, generation models must go beyond audio-visual realism to convey coherent character performance and other higher-level cinematic qualities. To fill this gap, we introduce MTAVG-Bench 2.0, a benchmark for diagnosing failure modes of cinematic expressiveness in multi-talker audio-video generation. Unlike prior settings that mainly focus on the quality of basic multi-turn dialogue, MTAVG-Bench 2.0 targets short-drama and scene-level generation, and establishes a high-level failure taxonomy spanning acting, narrative, atmosphere, and audio-visual language. Based on this taxonomy, we construct more than 10,000 question-answering evaluation instances, together with subsets for short-drama-level assessment and temporal localization of failure modes, to systematically evaluate the ability of omni large language models to diagnose high-level audio-visual failures. Experimental results show that commercial omni models such as Gemini substantially outperform other evaluators, yet even the strongest models continue to struggle with complex failures in our benchmark. These results demonstrate that MTAVG-Bench 2.0 provides a systematic benchmark for failure diagnosis in cinematic multi-talker audio-video generation.