# AVGen-Bench: A Task-Driven Benchmark for Multi-Granular Evaluation of Text-to-Audio-Video Generation

arXiv: https://arxiv.org/abs/2604.08540 · HTML: https://mlsanity.com/paper/2604.08540/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 36 references that are in the map.
- First posted on arXiv 9 Apr 2026.

## Builds on (most cited first, 23 of 23)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [Flow Matching for Generative Modeling](https://mlsanity.com/paper/2210.02747/) (2022)
- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [VBench: Comprehensive Benchmark Suite for Video Generative Models](https://mlsanity.com/paper/2311.17982/) (2023)
- [Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2304.08818/) (2023)
- [AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuning](https://mlsanity.com/paper/2307.04725/) (2023)
- [Towards Accurate Generative Models of Video: A New Metric & Challenges](https://mlsanity.com/paper/1812.01717/) (2018)
- [Large-scale Contrastive Language-Audio Pretraining with Feature Fusion and Keyword-to-Caption Augmentation](https://mlsanity.com/paper/2211.06687/) (2022)
- [Q-Align: Teaching LMMs for Visual Scoring via Discrete Text-Defined Levels](https://mlsanity.com/paper/2312.17090/) (2023)
- [VBench-2.0: Advancing Video Generation Benchmark Suite for Intrinsic Faithfulness](https://mlsanity.com/paper/2503.21755/) (2025)
- [LTX-2: Efficient Joint Audio-Visual Foundation Model](https://mlsanity.com/paper/2601.03233/) (2026)
- [VideoPhy-2: A Challenging Action-Centric Physical Commonsense Evaluation in Video Generation](https://mlsanity.com/paper/2503.06800/) (2025)
- [Meta Audiobox Aesthetics: Unified Automatic Quality Assessment for Speech, Music, and Sound](https://mlsanity.com/paper/2502.05139/) (2025)
- [MMAudio: Taming Multimodal Joint Training for High-Quality Video-to-Audio Synthesis](https://mlsanity.com/paper/2412.15322/) (2024)
- [Synchformer: Efficient Synchronization from Sparse Cues](https://mlsanity.com/paper/2401.16423/) (2024)
- [JavisDiT: Joint Audio-Video Diffusion Transformer with Hierarchical Spatio-Temporal Prior Synchronization](https://mlsanity.com/paper/2503.23377/) (2025)
- [PhyT2V: LLM-Guided Iterative Self-Refinement for Physics-Grounded Text-to-Video Generation](https://mlsanity.com/paper/2412.00596/) (2024)
- [UniVerse-1: Unified Audio-Video Generation via Stitching of Experts](https://mlsanity.com/paper/2509.06155/) (2025)
- [Kling-Foley: Multimodal Diffusion Transformer for High-Quality Video-to-Audio Generation](https://mlsanity.com/paper/2506.19774/) (2025)
- [A Lightweight Instrument-Agnostic Model for Polyphonic Note Transcription and Multipitch Estimation](https://mlsanity.com/paper/2203.09893/) (2022)
- [Audio-Sync Video Generation with Multi-Stream Temporal Control](https://mlsanity.com/paper/2506.08003/) (2025)
- [TTA-Bench: A Comprehensive Benchmark for Evaluating Text-to-Audio Models](https://mlsanity.com/paper/2509.02398/) (2025)

## Built on by (most cited first, 21 of 21)

- [MSAVBench: Towards Comprehensive and Reliable Evaluation of Multi-Shot Audio-Video Generation](https://mlsanity.com/paper/2605.20183/) (2026)
- [LongAV-Compass: Towards Unified Evaluation of Minute-Scale Audio-Visual Generation Across T2AV, I2AV, and V2AV](https://mlsanity.com/paper/2605.26244/) (2026)
- [CineDance: Towards Next-Generation Multi-Shot Long-Form Cinematic Audio-Video Generation](https://mlsanity.com/paper/2606.09639/) (2026)
- [MultiRef-Compass: Towards Comprehensive Evaluation of Multi-Reference-to-Audio-Video Generation](https://mlsanity.com/paper/2607.14189/) (2026)
- [EvalVerse: Pipeline-Aware and Expert-Calibrated Benchmarking for Professional Cinematic Video Generation](https://mlsanity.com/paper/2605.23271/) (2026)
- [VA-Judger: Reward Modeling from Human Preference Feedback for Joint Video-Audio Generation](https://mlsanity.com/paper/2608.18607/) (2026)
- [StreamAV-Bench: A Comprehensive Benchmark for Streaming Audio-Video Generation](https://mlsanity.com/paper/2608.26336/) (2026)
- [MTAVG-Bench 2.0: Diagnosing Failure Modes of Cinematic Expressiveness in Multi-Talker Audio-Video Generation](https://mlsanity.com/paper/2605.28035/) (2026)
- [KeyFrame-Compass: Towards Comprehensive Evaluation of Keyframe-Conditioned Video Generation](https://mlsanity.com/paper/2607.14202/) (2026)
- [Beyond Coherence: Benchmarking Professional Editing-Technique Execution in Multi-Shot Audio-Video Generation](https://mlsanity.com/paper/2609.08275/) (2026)
- [VTR-Bench: A Systematic Benchmark for Evaluating Visual Text Rendering in Video Generation](https://mlsanity.com/paper/2610.01499/) (2026)
- [TimeSteer: Inference-Time Speech Scheduling in Joint Audio-Visual Diffusion Models](https://mlsanity.com/paper/2609.01277/) (2026)
- [VeriPhy: Agentic Physical Reasoning for World Model Evaluation and Refinement](https://mlsanity.com/paper/2609.03153/) (2026)
- [AV-SafetyBench: A Safety Benchmark for Text-to-Audio-Video Generation](https://mlsanity.com/paper/2609.06991/) (2026)
- [All modalities are equal, but video is more equal: Closing the Cross-Attention Gap in Joint Video Generation](https://mlsanity.com/paper/2609.27901/) (2026)
- [Joint and Cross-Modal Video-Audio Generation and Editing: A Unified Formulation and Design Taxonomy](https://mlsanity.com/paper/2609.34381/) (2026)
- [Hear the World in Stereo: Learning Dynamic Spatial Correspondence for Immersive Joint Video-Audio Generation](https://mlsanity.com/paper/2609.38748/) (2026)
- [DMAD: Distribution Matching as Adversarial Distillation for Fast Visual Generation](https://mlsanity.com/paper/2610.02188/) (2026)
- [Kandinsky 6.0 Video: Foundation Models for Synchronized Video and Audio Generation](https://mlsanity.com/paper/2610.05608/) (2026)
- [TextVBench: Evaluating Text Rendering and Preservation in Video Generation](https://mlsanity.com/paper/2610.13412/) (2026)
- [SyncReward: Human-Aligned Synchronization Reward for Audio-Visual Generation](https://mlsanity.com/paper/2610.16338/) (2026)

## Abstract (arXiv)

Text-to-Audio-Video (T2AV) generation is rapidly becoming a core interface for media creation, yet its evaluation remains fragmented. Existing benchmarks largely assess audio and video in isolation or rely on coarse embedding similarity, failing to capture the fine-grained joint correctness required by realistic prompts. We introduce AVGen-Bench, a task-driven benchmark for T2AV generation featuring high-quality prompts across 11 real-world categories. To support comprehensive assessment, we propose a multi-granular evaluation framework that combines lightweight specialist models with Multimodal Large Language Models (MLLMs), enabling evaluation from perceptual quality to fine-grained semantic controllability. Our evaluation reveals a pronounced gap between strong audio-visual aesthetics and weak semantic reliability, including persistent failures in text rendering, speech coherence, physical reasoning, and a universal breakdown in musical pitch control. Code and benchmark resources are available at http://aka.ms/avgenbench.