mlsanityData to 7 Oct 2026

StrixAE: An Intelligent Agent for Audio Enhancement under Complex Distortion Coupling in Real-World Scenarios

arXiv 2609.03414 ↗First posted 3 Sep 2026cs.SD cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Audio enhancement in real-world scenarios involves complex distortion couplings and requires personalized enhancement. Existing solutions struggle to address both simultaneously. To improve robustness and enable autonomous operation in such scenarios, we propose StrixAE, an agent based on a multimodal large language model (MLLM). StrixAE leverages the MLLM as a controller to coordinate multiple audio enhancement and personalization models. To further enhance system robustness, reduce artifacts, and improve generalization across diverse real-world scenarios, StrixAE is trained through a two-stage process: first, CoT supervised fine-tuning on AcoustBench to ground basic reasoning and tool invocation; second, Audio Perception Reinforcement Learning (APRL), a reward design specifically tailored for audio restoration pipelines that jointly optimizes format validity, structural coherence, and perceptual quality. Unlike generic RL fine-tuning, APRL introduces structured rewards that enforce executable pipelines and logical section ordering, enabling the agent to produce reliable, interpretable enhancement plans without hallucinated tools. Based on real-world test datasets, our proposed method outperforms most existing open-source and proprietary solutions, achieving state-of-the-art performance across multiple perceptual metrics and demonstrating strong generalization robustness.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 40 of its 54 references that are in the map. First posted on arXiv 3 Sep 2026.

Builds on

40 of 54 references.

References · 40

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
  3. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  4. 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Frameworkcited by 1,761
  5. 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,302
  6. 2023HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Facecited by 1,173
  7. 2022Scaling Laws for Reward Model Overoptimizationcited by 1,056
  8. 2023MM-REACT: Prompting ChatGPT for Multimodal Reasoning and Actioncited by 511
  9. 2025Qwen3-Omni Technical Reportcited by 451
  10. 2025DanceGRPO: Unleashing GRPO on Visual Generationcited by 388
  11. 2022UTMOS: UTokyo-SaruLab System for VoiceMOS Challenge 2022cited by 381
  12. 2023GPT4Tools: Teaching Large Language Model to Use Tools via Self-instructioncited by 245
28 more
  1. 2025EmbodiedBench: Comprehensive Benchmarking Multi-modal Large Language Models for Vision-Driven Embodied Agentscited by 213
  2. 2021NISQA: A Deep CNN-Self-Attention Model for Multidimensional Speech Quality Prediction with Crowdsourced Datasetscited by 189
  3. 2021DNSMOS P.835: A Non-Intrusive Perceptual Objective Speech Quality Metric to Evaluate Noise Suppressorscited by 165
  4. 2020The INTERSPEECH 2020 Deep Noise Suppression Challenge: Datasets, Subjective Testing Framework, and Challenge Resultscited by 128
  5. 2025Audio-Reasoner: Improving Reasoning Capability in Large Audio Language Modelscited by 94
  6. 2022TF-GridNet: Integrating Full- and Sub-Band Modeling for Speech Separationcited by 62
  7. 2023ICASSP 2023 Deep Noise Suppression Challengecited by 62
  8. 2024An Investigation of Incorporating Mamba for Speech Enhancementcited by 61
  9. 2024Open-Source Conversational AI with SpeechBrain 1.0cited by 55
  10. 2023See and Think: Embodied Agent in Virtual Environmentcited by 54
  11. 2023MP-SENet: A Speech Enhancement Model with Parallel Denoising of Magnitude and Phase Spectracited by 51
  12. 2022CMGAN: Conformer-based Metric GAN for Speech Enhancementcited by 45
  13. 2024From Multimodal LLMs to Generalist Embodied Agents: Methods and Lessonscited by 42
  14. 2023MossFormer2: Combining Transformer and RNN-Free Recurrent Network for Enhanced Time-Domain Monaural Speech Separationcited by 39
  15. 2025JarvisArt: Liberating Human Artistic Creativity via an Intelligent Photo Retouching Agentcited by 39
  16. 2025JarvisIR: Elevating Autonomous Driving Perception with Intelligent Image Restorationcited by 32
  17. 2026Gen-Searcher: Reinforcing Agentic Search for Image Generationcited by 25
  18. 2024SCOREQ: Speech Quality Assessment with Contrastive Regressioncited by 23
  19. 2023Explicit Estimation of Magnitude and Phase Spectra in Parallel for High-Quality Speech Enhancementcited by 13
  20. 2024DiffAgent: Fast and Accurate Text-to-Image API Selection with Large Language Modelcited by 12
  21. 2025Interspeech 2025 URGENT Speech Enhancement Challengecited by 12
  22. 2024TIGER: Time-frequency Interleaved Gain Extraction and Reconstruction for Efficient Speech Separationcited by 10
  23. 2024Mamba-SEUNet: Mamba UNet for Monaural Speech Enhancementcited by 10
  24. 2025ClearerVoice-Studio: Bridging Advanced Speech Processing Research and Practical Deploymentcited by 10
  25. 2023TEA-PSE 3.0: Tencent-Ethereal-Audio-Lab Personalized Speech Enhancement System For ICASSP 2023 DNS Challengecited by 8
  26. 2024An Analysis of the Variance of Diffusion-based Speech Enhancementcited by 5
  27. 2026ICASSP 2026 URGENT Speech Enhancement Challengecited by 4
  28. 2025Lessons Learned from the URGENT 2024 Speech Enhancement Challengecited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026AuTAgent: A Reinforcement Learning Framework for Tool-Augmented Audio Reasoning
  2. 2026EvoAudio: Recursive Self-Improvement for Audio Understanding
  3. 2025AudioGenie: A Training-Free Multi-Agent Framework for Diverse Multimodality-to-Multiaudio Generation
  4. 2026AudioRouter: Data Efficient Audio Understanding via RL based Dual Reasoning
  5. 2025SoundMind: RL-Incentivized Logic Reasoning for Audio-Language Models
  6. 2025Reinforcement Learning Outperforms Supervised Fine-Tuning: A Case Study on Audio Question Answering
  7. 2024Audio-Agent: Leveraging LLMs For Audio Generation, Editing and Composition
  8. 2025Audio Palette: A Diffusion Transformer with Multi-Signal Conditioning for Controllable Foley Synthesis