# AQA-TTRL: Self-Adaptation in Audio Question Answering with Test-Time Reinforcement Learning

arXiv: https://arxiv.org/abs/2510.05478 · HTML: https://mlsanity.com/paper/2510.05478/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Oct 2025.

## Built on by (most cited first, 9 of 9)

- [Audio-Zero: Label-Free Self-Evolution for Fine-Grained Audio Reasoning](https://mlsanity.com/paper/2607.20166/) (2026)
- [Dual Consensus: Escaping from Spurious Majority in Unsupervised RLVR via Two-Stage Vote Mechanism](https://mlsanity.com/paper/2603.16223/) (2026)
- [EvoAudio: Recursive Self-Improvement for Audio Understanding](https://mlsanity.com/paper/2609.27389/) (2026)
- [CALM: Class-Conditional Sparse Attention Vectors for Large Audio-Language Models](https://mlsanity.com/paper/2602.07077/) (2026)
- [Med-Evo: Test-time Self-evolution for Medical Multimodal Large Language Models](https://mlsanity.com/paper/2603.07443/) (2026)
- [MAPLE: Elevating Medical Reasoning from Statistical Consensus to Process-Led Alignment](https://mlsanity.com/paper/2603.08987/) (2026)
- [Do Audio LLMs Listen Before They Act? Diagnosing Acoustic-Context Gating in Voice Agents](https://mlsanity.com/paper/2609.32536/) (2026)
- [AdaLoop: Adaptive-Depth Latent Reasoning for Audio Language Models](https://mlsanity.com/paper/2610.06949/) (2026)
- [SkillFormer: Skill-Decomposed Adaptation for Audio Language Models](https://mlsanity.com/paper/2610.07533/) (2026)

## Abstract (arXiv)

Large Audio Language Models (LALMs) exhibit strong capabilities in general audio understanding but remain static after deployment, limiting their adaptability to real-world data. Since supervised fine-tuning is costly, we propose AQA-TTRL, a novel framework for audio understanding that enables on-the-fly evolution via test-time reinforcement learning using only unlabeled test data. It generates pseudo-labels via majority voting and optimizes the model through reinforcement learning. To address the noise in self-generated labels, we introduce confidence weighting to adjust training signals. Furthermore, multiple-attempt sampling mitigates advantage collapse and stabilizes training. Across MMAU, MMAR, and MMSU, AQA-TTRL achieves significant average improvements of 4.42% for Qwen2.5-Omni 7B and 11.04% for the 3B model. Notably, the adapted 3B model outperforms direct inference of the unadapted 7B model, highlighting the effectiveness of test-time adaptation in audio understanding.