# Domain Adaptation Method and Modality Gap Impact in Audio-Text Models for Prototypical Sound Classification

arXiv: https://arxiv.org/abs/2506.04376 · HTML: https://mlsanity.com/paper/2506.04376/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 22 references that are in the map.
- First posted on arXiv 4 Jun 2025.

## Builds on (most cited first, 12 of 12)

- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [Large-scale Contrastive Language-Audio Pretraining with Feature Fusion and Keyword-to-Caption Augmentation](https://mlsanity.com/paper/2211.06687/) (2022)
- [Mind the Gap: Understanding the Modality Gap in Multi-modal Contrastive Representation Learning](https://mlsanity.com/paper/2203.02053/) (2022)
- [AudioCLIP: Extending CLIP to Image, Text and Audio](https://mlsanity.com/paper/2106.13043/) (2021)
- [A multi-device dataset for urban acoustic scene classification](https://mlsanity.com/paper/1807.09840/) (2018)
- [Audio-Text Models Do Not Yet Leverage Natural Language](https://mlsanity.com/paper/2303.10667/) (2023)
- [It's Not a Modality Gap: Characterizing and Addressing the Contrastive Gap](https://mlsanity.com/paper/2405.18570/) (2024)
- [Weakly-supervised Automated Audio Captioning via text only training](https://mlsanity.com/paper/2309.12242/) (2023)
- [Domain Adaptation for Contrastive Audio-Language Models](https://mlsanity.com/paper/2402.09585/) (2024)
- [PALM: Few-Shot Prompt Learning for Audio Language Models](https://mlsanity.com/paper/2409.19806/) (2024)
- [A Multimodal Prototypical Approach for Unsupervised Sound Classification](https://mlsanity.com/paper/2306.12300/) (2023)
- [Multi-label Zero-Shot Audio Classification with Temporal Attention](https://mlsanity.com/paper/2409.00408/) (2024)

## Built on by (most cited first, 2 of 2)

- [Drift-Augmented Scoring: Text-Derived Noise Robustness for Zero-Shot Audio-Language Classification](https://mlsanity.com/paper/2606.04844/) (2026)
- [Prototype-Rectified Iterative Self-supervised Manifold Denoising under Severe Acoustic Shift](https://mlsanity.com/paper/2608.15037/) (2026)

## Abstract (arXiv)

Audio-text models are widely used in zero-shot environmental sound classification as they alleviate the need for annotated data. However, we show that their performance severely drops in the presence of background sound sources. Our analysis reveals that this degradation is primarily driven by SNR levels of background soundscapes, and independent of background type. To address this, we propose a novel method that quantifies and integrates the contribution of background sources into the classification process, improving performance without requiring model retraining. Our domain adaptation technique enhances accuracy across various backgrounds and SNR conditions. Moreover, we analyze the modality gap between audio and text embeddings, showing that narrowing this gap improves classification performance. The method generalizes effectively across state-of-the-art prototypical approaches, showcasing its scalability and robustness for diverse environments.