# AVBench: Human-Aligned and Automated Evaluation Benchmark for Audio-Video Generative Models

arXiv: https://arxiv.org/abs/2605.24652 · HTML: https://mlsanity.com/paper/2605.24652/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion and multimodal model alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation.
- Builds on 28 of its 35 references that are in the map.
- First posted on arXiv 23 May 2026.

## Placed in

- LLM alignment / RLHF / preference optimisation / [Diffusion and multimodal model alignment](https://mlsanity.com/alignment/diffusion-and-multimodal-model-alignment/)

## Builds on (most cited first, 28 of 28)

- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [LAION-5B: An open large-scale dataset for training next generation image-text models](https://mlsanity.com/paper/2210.08402/) (2022)
- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [Wan: Open and Advanced Large-Scale Video Generative Models](https://mlsanity.com/paper/2503.20314/) (2025)
- [VBench: Comprehensive Benchmark Suite for Video Generative Models](https://mlsanity.com/paper/2311.17982/) (2023)
- [ImageBind: One Embedding Space To Bind Them All](https://mlsanity.com/paper/2305.05665/) (2023)
- [AudioLDM: Text-to-Audio Generation with Latent Diffusion Models](https://mlsanity.com/paper/2301.12503/) (2023)
- [Qwen2-Audio Technical Report](https://mlsanity.com/paper/2407.10759/) (2024)
- [InternVid: A Large-scale Video-Text Dataset for Multimodal Understanding and Generation](https://mlsanity.com/paper/2307.06942/) (2023)
- [Qwen3-Omni Technical Report](https://mlsanity.com/paper/2509.17765/) (2025)
- [Exploring Video Quality Assessment on User Generated Contents from Aesthetic and Technical Perspectives](https://mlsanity.com/paper/2211.04894/) (2022)
- [NISQA: A Deep CNN-Self-Attention Model for Multidimensional Speech Quality Prediction with Crowdsourced Datasets](https://mlsanity.com/paper/2104.09494/) (2021)
- [Taming Visually Guided Sound Generation](https://mlsanity.com/paper/2110.08791/) (2021)
- [Audiobox: Unified Audio Generation with Natural Language Prompts](https://mlsanity.com/paper/2312.15821/) (2023)
- [Ovi: Twin Backbone Cross-Modal Fusion for Audio-Video Generation](https://mlsanity.com/paper/2510.01284/) (2025)
- [Seedance 1.5 pro: A Native Audio-Visual Joint Generation Foundation Model](https://mlsanity.com/paper/2512.13507/) (2025)
- [OpenHumanVid: A Large-Scale High-Quality Dataset for Enhancing Human-Centric Video Generation](https://mlsanity.com/paper/2412.00115/) (2024)
- [LatentSync: Taming Audio-Conditioned Latent Diffusion Models for Lip Sync with SyncNet Supervision](https://mlsanity.com/paper/2412.09262/) (2024)
- [HuMo: Human-Centric Video Generation via Collaborative Multi-Modal Conditioning](https://mlsanity.com/paper/2509.08519/) (2025)
- [HunyuanVideo-Foley: Multimodal Diffusion with Representation Alignment for High-Fidelity Foley Audio Generation](https://mlsanity.com/paper/2508.16930/) (2025)
- [VABench: A Comprehensive Benchmark for Audio-Video Generation](https://mlsanity.com/paper/2512.09299/) (2025)
- [T2AV-Compass: Towards Unified Evaluation for Text-to-Audio-Video Generation](https://mlsanity.com/paper/2512.21094/) (2025)
- [MMDisCo: Multi-Modal Discriminator-Guided Cooperative Diffusion for Joint Audio and Video Generation](https://mlsanity.com/paper/2405.17842/) (2024)
- [AV-Link: Temporally-Aligned Diffusion Features for Cross-Modal Audio-Video Generation](https://mlsanity.com/paper/2412.15191/) (2024)
- [Speech DF Arena: A Leaderboard for Speech DeepFake Detection Models](https://mlsanity.com/paper/2509.02859/) (2025)
- [DreamID-Omni: Unified Framework for Controllable Human-Centric Audio-Video Generation](https://mlsanity.com/paper/2602.12160/) (2026)
- [JointAVBench: A Benchmark for Joint Audio-Visual Reasoning Evaluation](https://mlsanity.com/paper/2512.12772/) (2025)

## Built on by (most cited first, 5 of 5)

- [MultiRef-Compass: Towards Comprehensive Evaluation of Multi-Reference-to-Audio-Video Generation](https://mlsanity.com/paper/2607.14189/) (2026)
- [OmniVR: Audio-Video Conditional Generation for Archival Footage Restoration](https://mlsanity.com/paper/2608.04224/) (2026)
- [VTR-Bench: A Systematic Benchmark for Evaluating Visual Text Rendering in Video Generation](https://mlsanity.com/paper/2610.01499/) (2026)
- [Human-Centric Intelligence in the Era of Foundation Models: A Survey](https://mlsanity.com/paper/2608.18184/) (2026)
- [What Do Audio-Visual Synchronization Metrics Actually Measure?](https://mlsanity.com/paper/2608.25157/) (2026)

## Abstract (arXiv)

Rapid advances in audio-video (AV) generation have enabled high-fidelity synthesis with synchronized sound, particularly for human-related scenarios involving speech and interactions. Yet evaluation for AV generation remains at an early stage, with only a few coarse-grained benchmarks for human-related scenarios and relying on limited preset evaluations with generic multimodal LLMs, leading to inaccurate assessments of model capabilities. To address these issues, we introduce AVBench, a fully automated benchmark tailored for human-centric AV generation. AVBench is built on two key designs for comprehensive and accurate evaluation: (i) Human-centric and fine-grained metrics. AVBench integrates ten evaluation dimensions designed for human-centered real-world scenarios, covering visual quality, audio quality, and multi-level consistency across modalities. These practical metrics capture human-related details that existing benchmarks often overlook. (ii) Specialized evaluators via preference learning. To address the lack of specialized training data, we construct large-scale supervision by transforming real-world videos into diverse training pairs with controlled perturbations. After fine-tuning on this high-quality dataset, the evaluators learn to reliably detect subtle cross-modal inconsistencies. Crucially, instead of producing discrete textual judgment, AVBench derives continuous evaluation scores from the model's prediction confidence on binary decisions. This probabilistic scoring mechanism enables a more reliable assessment than traditional VQA-style evaluation and aligns closely with human judgment. Taken together, AVBench offers automated evaluation for AV generation, demonstrates strong potential for data filtering, and serves as a differentiable reward signal for Reinforcement Learning from Human Feedback (RLHF).