# Audio-Oscar: A Multi-Agent System for Complex Audio Scene Generation, Orchestration, and Refinement

arXiv: https://arxiv.org/abs/2606.07397 · HTML: https://mlsanity.com/paper/2606.07397/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Jun 2026.

## Built on by (most cited first, 4 of 4)

- [UI-MOPD: Multi-Platform On-Policy Distillation for Unified GUI Agents](https://mlsanity.com/paper/2607.04425/) (2026)
- [AuK Technical Report: An Open-Source Foundational Model for Speech Generation and Editing](https://mlsanity.com/paper/2609.08936/) (2026)
- [OmniVChat: Synthesizing, Benchmarking, and Training for Native Audio-Visual Dialogue](https://mlsanity.com/paper/2609.21465/) (2026)
- [From Script to Drama: An Agentic Framework for Controllable Multi-Speaker Dialogue TTS](https://mlsanity.com/paper/2609.33362/) (2026)

## Abstract (arXiv)

In recent years, audio generation has made significant progress in tasks such as text-to-speech (TTS), text-to-audio (TTA) and text-to-music (TTM). However, generating long-form and controllable audio from complex audio scene descriptions remains a significant challenge, as such scenes often require coordinated speech, sound effects, music, songs, temporal structure, and post-production. In this work, we introduce Audio-Oscar, a multi-agent framework for generating audio from complex descriptions. Audio-Oscar coordinates a set of specialist agents, each responsible for a different aspect of the audio scene, including character modeling and voice design, speech generation, fine-grained timeline planning, model selection, non-speech generation, and audio post-production. Audio-Oscar further incorporates feedback-driven refinement. In addition, to address the lack of suitable benchmarks for evaluating audio generation from complex audio scene descriptions, we construct ASG-Bench, an Audio Scene Generation Benchmark containing both scene descriptions paired with reference audio and text-only scene descriptions. Each scene is annotated with target audio events and temporal statements to evaluate whether the generated audio faithfully realizes the required scene content and temporal structure. Experimental results show that Audio-Oscar can effectively generate audio that matches complex scene descriptions. Project samples are available at https://audiooscar.github.io/. Our code is available at https://github.com/ziye26/Audio-Oscar.