mlsanityData to 14 Oct 2026

QAMRO: Quality-aware Adaptive Margin Ranking Optimization for Human-aligned Assessment of Audio Generation Systems

arXiv 2508.08957 ↗First posted 12 Aug 2025cs.SD cs.AI cs.LGcited by 3

Not on the map. Why?

Abstract (arXiv)

Evaluating audio generation systems, including text-to-music (TTM), text-to-speech (TTS), and text-to-audio (TTA), remains challenging due to the subjective and multi-dimensional nature of human perception. Existing methods treat mean opinion score (MOS) prediction as a regression problem, but standard regression losses overlook the relativity of perceptual judgments. To address this limitation, we introduce QAMRO, a novel Quality-aware Adaptive Margin Ranking Optimization framework that seamlessly integrates regression objectives from different perspectives, aiming to highlight perceptual differences and prioritize accurate ratings. Our framework leverages pre-trained audio-text models such as CLAP and Audiobox-Aesthetics, and is trained exclusively on the official AudioMOS Challenge 2025 dataset. It demonstrates superior alignment with human evaluations across all dimensions, significantly outperforming robust baseline models.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 43 of its 50 references that are in the map. First posted on arXiv 12 Aug 2025.

Builds on

43 of 50 references.

References · 43

  1. 2016An overview of gradient descent optimization algorithmscited by 1,033
  2. 2022Large-scale Contrastive Language-Audio Pretraining with Feature Fusion and Keyword-to-Caption Augmentationcited by 719
  3. 2023AudioLDM: Text-to-Audio Generation with Latent Diffusion Modelscited by 599
  4. 2023Simple and Controllable Music Generationcited by 488
  5. 2023MusicLM: Generating Music From Textcited by 448
  6. 2022UTMOS: UTokyo-SaruLab System for VoiceMOS Challenge 2022cited by 390
  7. 2024F5-TTS: A Fairytaler that Fakes Fluent and Faithful Speech with Flow Matchingcited by 361
  8. 2023Make-An-Audio: Text-To-Audio Generation with Prompt-Enhanced Diffusion Modelscited by 324
  9. 2021Post-Training Quantization for Vision Transformercited by 278
  10. 2018A Note on the Inception Scorecited by 226
  11. 2024NaturalSpeech 3: Zero-Shot Speech Synthesis with Factorized Codec and Diffusion Modelscited by 226
  12. 2018Fr'echet Audio Distance: A Metric for Evaluating Music Enhancement Algorithmscited by 225
31 more
  1. 2023Noise2Music: Text-conditioned Music Generation with Diffusion Modelscited by 173
  2. 2021DNSMOS P.835: A Non-Intrusive Perceptual Objective Speech Quality Metric to Evaluate Noise Suppressorscited by 172
  3. 2025Meta Audiobox Aesthetics: Unified Automatic Quality Assessment for Speech, Music, and Soundcited by 137
  4. 2024VALL-E 2: Neural Codec Language Models are Human Parity Zero-Shot Text to Speech Synthesizerscited by 122
  5. 2023SoundStorm: Efficient Parallel Audio Generationcited by 114
  6. 2023Mousai: Text-to-Music Generation with Long-Context Latent Diffusioncited by 78
  7. 2019Ranked List Loss for Deep Metric Learningcited by 77
  8. 2021Generalization Ability of MOS Prediction Networkscited by 69
  9. 2023MuseCoco: Generating Symbolic Music from Textcited by 52
  10. 2024Masked Audio Generation using a Single Non-Autoregressive Transformercited by 50
  11. 2019SoDeep: a Sorting Deep net to learn ranking loss surrogatescited by 28
  12. 2022Multitrack Music Transformercited by 28
  13. 2020PiRank: Scalable Learning To Rank via Differentiable Sortingcited by 27
  14. 2022Rank-based Decomposable Losses in Machine Learning: A Surveycited by 26
  15. 2020A Ranking-based, Balanced Loss Function Unifying Classification and Localisation in Object Detectioncited by 25
  16. 2019DR Loss: Improving Object Detection by Distributional Rankingcited by 24
  17. 2021Rank & Sort Loss for Object Detection and Instance Segmentationcited by 22
  18. 2022Joint Optimization of Ranking and Calibration with Contextualized Hybrid Modelcited by 22
  19. 2021Utilizing Self-supervised Representations for MOS Predictioncited by 20
  20. 2025MusicEval: A Generative Music Dataset with Expert Ratings for Automatic Text-to-Music Evaluationcited by 18
  21. 2019An Alternative Cross Entropy Loss for Learning-to-Rankcited by 17
  22. 2023ERNIE-Music: Text-to-Waveform Music Generation with Diffusion Modelscited by 15
  23. 2024Offline Model-Based Optimization by Learning to Rankcited by 13
  24. 2024Enhancing Knowledge Distillation of Large Language Models through Efficient Multi-Modal Distribution Alignmentcited by 11
  25. 2022Benchmarking Deep AUROC Optimization: Loss Functions and Algorithmic Choicescited by 10
  26. 2021Rethinking and Reweighting the Univariate Losses for Multi-Label Ranking: Consistency and Generalizationcited by 9
  27. 2024Self-Supervised Speech Quality Estimation and Enhancement Using Only Clean Speechcited by 7
  28. 2023A Unified Framework for Rank-based Loss Minimizationcited by 3
  29. 2024Pairwise Ranking Loss for Multi-Task Learning in Recommender Systemscited by 3
  30. 2024Bucketed Ranking-based Losses for Efficient Training of Object Detectorscited by 3
  31. 2025Towards Robust Assessment of Pathological Voices via Combined Low-Level Descriptors and Foundation Model Representationscited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026CMI-RewardBench: Evaluating Music Reward Models with Compositional Multimodal Instructioncited by 5
  2. 2026Robust Generative Audio Quality Assessment: Disentangling Quality from Spurious Correlationscited by 0
  3. 2026DeRA-MOS: Optimizing Text-to-Music Evaluation via Decoupled Listwise Ranking and Modality Alignmentcited by 0