# On Metric Learning for Audio-Text Cross-Modal Retrieval

arXiv: https://arxiv.org/abs/2203.15537 · HTML: https://mlsanity.com/paper/2203.15537/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Mar 2022.

## Built on by (most cited first, 30 of 30)

- [WavCaps: A ChatGPT-Assisted Weakly-Labelled Audio Captioning Dataset for Audio-Language Multimodal Research](https://mlsanity.com/paper/2303.17395/) (2023)
- [ONE-PEACE: Exploring One General Representation Model Toward Unlimited Modalities](https://mlsanity.com/paper/2305.11172/) (2023)
- [Separate Anything You Describe](https://mlsanity.com/paper/2308.05037/) (2023)
- [CompA: Addressing the Gap in Compositional Reasoning in Audio-Language Models](https://mlsanity.com/paper/2310.08753/) (2023)
- [A Survey of Multimodal Retrieval-Augmented Generation](https://mlsanity.com/paper/2504.08748/) (2025)
- [Audio Retrieval with WavText5K and CLAP Training](https://mlsanity.com/paper/2209.14275/) (2022)
- [Mellow: a small audio language model for reasoning](https://mlsanity.com/paper/2503.08540/) (2025)
- [Automated Audio Captioning: An Overview of Recent Progress and New Challenges](https://mlsanity.com/paper/2205.05949/) (2022)
- [Modality-Independent Teachers Meet Weakly-Supervised Audio-Visual Event Parser](https://mlsanity.com/paper/2305.17343/) (2023)
- [T-CLAP: Temporal-Enhanced Contrastive Language-Audio Pretraining](https://mlsanity.com/paper/2404.17806/) (2024)
- [Visually-Aware Audio Captioning With Adaptive Audio-Visual Attention](https://mlsanity.com/paper/2210.16428/) (2022)
- [ReCLAP: Improving Zero Shot Audio Classification by Describing Sounds](https://mlsanity.com/paper/2409.09213/) (2024)
- [Exploring Train and Test-Time Augmentations for Audio-Language Learning](https://mlsanity.com/paper/2210.17143/) (2022)
- [Data-Efficient Multimodal Fusion on a Single GPU](https://mlsanity.com/paper/2312.10144/) (2023)
- [Leveraging Pre-trained AudioLDM for Sound Generation: A Benchmark Study](https://mlsanity.com/paper/2303.03857/) (2023)
- [Revisiting Deep Audio-Text Retrieval Through the Lens of Transportation](https://mlsanity.com/paper/2405.10084/) (2024)
- [TACOS: Temporally-aligned Audio CaptiOnS for Language-Audio Pretraining](https://mlsanity.com/paper/2505.07609/) (2025)
- [FLAP: Fast Language-Audio Pre-training](https://mlsanity.com/paper/2311.01615/) (2023)
- [Killing two birds with one stone: Can an audio captioning system also be used for audio-text retrieval?](https://mlsanity.com/paper/2308.15090/) (2023)
- [Mitigating Audiovisual Mismatch in Visual-Guide Audio Captioning](https://mlsanity.com/paper/2505.22045/) (2025)
- [SupCLAP: Controlling Optimization Trajectory Drift in Audio-Text Contrastive Learning with Support Vector Regularization](https://mlsanity.com/paper/2509.21033/) (2025)
- [CLARA: Multilingual Contrastive Learning for Audio Representation Acquisition](https://mlsanity.com/paper/2310.11830/) (2023)
- [Fusing Audio and Metadata Embeddings Improves Language-based Audio Retrieval](https://mlsanity.com/paper/2406.15897/) (2024)
- [Estimated Audio-Caption Correspondences Improve Language-Based Audio Retrieval](https://mlsanity.com/paper/2408.11641/) (2024)
- [DiffATR: Diffusion-based Generative Modeling for Audio-Text Retrieval](https://mlsanity.com/paper/2409.10025/) (2024)
- [Data leakage in cross-modal retrieval training: A case study](https://mlsanity.com/paper/2302.12258/) (2023)
- [Start from Video-Music Retrieval: An Inter-Intra Modal Loss for Cross Modal Retrieval](https://mlsanity.com/paper/2407.19415/) (2024)
- [Enhancing Audio-Language Models through Self-Supervised Post-Training with Text-Audio Pairs](https://mlsanity.com/paper/2408.09269/) (2024)
- [SNeL: A Structured Neuro-Symbolic Language for Entity-Based Multimodal Scene Understanding](https://mlsanity.com/paper/2306.06036/) (2023)
- [ASK: Adaptive Self-improving Knowledge Framework for Audio Text Retrieval](https://mlsanity.com/paper/2512.19703/) (2025)

## Abstract (arXiv)

Audio-text retrieval aims at retrieving a target audio clip or caption from a pool of candidates given a query in another modality. Solving such cross-modal retrieval task is challenging because it not only requires learning robust feature representations for both modalities, but also requires capturing the fine-grained alignment between these two modalities. Existing cross-modal retrieval models are mostly optimized by metric learning objectives as both of them attempt to map data to an embedding space, where similar data are close together and dissimilar data are far apart. Unlike other cross-modal retrieval tasks such as image-text and video-text retrievals, audio-text retrieval is still an unexplored task. In this work, we aim to study the impact of different metric learning objectives on the audio-text retrieval task. We present an extensive evaluation of popular metric learning objectives on the AudioCaps and Clotho datasets. We demonstrate that NT-Xent loss adapted from self-supervised learning shows stable performance across different datasets and training settings, and outperforms the popular triplet-based losses. Our code is available at https://github.com/XinhaoMei/audio-text_retrieval.