mlsanityData to 14 Oct 2026

Challenge on Sound Scene Synthesis: Evaluating Text-to-Audio Generation

arXiv 2410.17589 ↗First posted 23 Oct 2024cs.SD cs.AI cs.LGcited by 6

Not on the map. Why?

Abstract (arXiv)

Despite significant advancements in neural text-to-audio generation, challenges persist in controllability and evaluation. This paper addresses these issues through the Sound Scene Synthesis challenge held as part of the Detection and Classification of Acoustic Scenes and Events 2024. We present an evaluation protocol combining objective metric, namely Fr'echet Audio Distance, with perceptual assessments, utilizing a structured prompt format to enable diverse captions and effective evaluation. Our analysis reveals varying performance across sound categories and model architectures, with larger models generally excelling but innovative lightweight approaches also showing promise. The strong correlation between objective metrics and human ratings validates our evaluation approach. We discuss outcomes in terms of audio quality, controllability, and architectural considerations for text-to-audio synthesizers, providing direction for future research.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 19 of its 26 references that are in the map. First posted on arXiv 23 Oct 2024.

Builds on

19 of 26 references.

References · 19

  1. 2016CNN Architectures for Large-Scale Audio Classificationcited by 704
  2. 2023AudioLDM: Text-to-Audio Generation with Latent Diffusion Modelscited by 599
  3. 2019PANNs: Large-Scale Pretrained Audio Neural Networks for Audio Pattern Recognitioncited by 574
  4. 2019Clotho: An Audio Captioning Datasetcited by 358
  5. 2023AudioLDM 2: Learning Holistic Audio Generation with Self-supervised Pretrainingcited by 344
  6. 2023Make-An-Audio: Text-To-Audio Generation with Prompt-Enhanced Diffusion Modelscited by 324
  7. 2022AudioGen: Textually Guided Audio Generationcited by 295
  8. 2023WavCaps: A ChatGPT-Assisted Weakly-Labelled Audio Captioning Dataset for Audio-Language Multimodal Researchcited by 267
  9. 2018Fr'echet Audio Distance: A Metric for Evaluating Music Enhancement Algorithmscited by 225
  10. 2024Stable Audio Opencited by 197
  11. 2019Effectively Unbiased FID and Inception Score and where to find themcited by 165
  12. 2024Tango 2: Aligning Diffusion-based Text-to-Audio Generations through Direct Preference Optimizationcited by 97
7 more
  1. 2023Adapting Frechet Audio Distance for Generative Music Evaluationcited by 92
  2. 2024Auffusion: Leveraging the Power of Diffusion and Large Language Models for Text-to-Audio Generationcited by 54
  3. 2023ConsistencyTTA: Accelerating Diffusion-Based Text-to-Audio Generation with Consistency Distillationcited by 22
  4. 2023Foley Sound Synthesis at the DCASE 2023 Challengecited by 19
  5. 2024Improving Text-To-Audio Models with Synthetic Captionscited by 19
  6. 2024Correlation of Fr'echet Audio Distance With Human Perception of Environmental Audio Is Embedding Dependantcited by 15
  7. 2022A Proposal for Foley Sound Synthesis Challengecited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 3.

6 papers.

  1. 2025KAD: No More FAD! An Effective and Efficient Evaluation Metric for Audio Generationcited by 24
  2. 2025Hear What Matters! Text-conditioned Selective Video-to-Audio Generationcited by 3
  3. 2025Mind the Prompt: Prompting Strategies in Audio Generations for Improving Sound Classificationcited by 1
  4. 2026MoEScore: Mixture-of-Experts-Based Text-Audio Relevance Score Prediction for Text-to-Audio System Evaluationcited by 0
  5. 2026Investigating Group Relative Policy Optimization for Diffusion Transformer based Text-to-Audio Generationcited by 0
  6. 2026SsgCaps: A controlled dataset for the evaluation of sound scene generation algorithmscited by 0