# Audio-text Retrieval in Context

arXiv: https://arxiv.org/abs/2203.13645 · HTML: https://mlsanity.com/paper/2203.13645/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 19 references that are in the map.
- First posted on arXiv 25 Mar 2022.

## Builds on (most cited first, 13 of 13)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Deep Visual-Semantic Alignments for Generating Image Descriptions](https://mlsanity.com/paper/1412.2306/) (2014)
- [NetVLAD: CNN architecture for weakly supervised place recognition](https://mlsanity.com/paper/1511.07247/) (2015)
- [CNN Architectures for Large-Scale Audio Classification](https://mlsanity.com/paper/1609.09430/) (2016)
- [VGGSound: A Large-scale Audio-Visual Dataset](https://mlsanity.com/paper/2004.14368/) (2020)
- [PANNs: Large-Scale Pretrained Audio Neural Networks for Audio Pattern Recognition](https://mlsanity.com/paper/1912.10211/) (2019)
- [Clotho: An Audio Captioning Dataset](https://mlsanity.com/paper/1910.09387/) (2019)
- [Use What You Have: Video Retrieval Using Representations From Collaborative Experts](https://mlsanity.com/paper/1907.13487/) (2019)
- [Learning a Text-Video Embedding from Incomplete and Heterogeneous Data](https://mlsanity.com/paper/1804.02516/) (2018)
- [Deep Fragment Embeddings for Bidirectional Image Sentence Mapping](https://mlsanity.com/paper/1406.5679/) (2014)
- [Learnable pooling with Context Gating for video classification](https://mlsanity.com/paper/1706.06905/) (2017)
- [Audio Retrieval with Natural Language Queries](https://mlsanity.com/paper/2105.02192/) (2021)

## Built on by (most cited first, 10 of 10)

- [Retrieving Multimodal Information for Augmented Generation: A Survey](https://mlsanity.com/paper/2303.10868/) (2023)
- [Separate Anything You Describe](https://mlsanity.com/paper/2308.05037/) (2023)
- [Exposing and Mitigating Spurious Correlations for Cross-Modal Retrieval](https://mlsanity.com/paper/2304.03391/) (2023)
- [Zero-shot audio captioning with audio-language model guidance and audio context keywords](https://mlsanity.com/paper/2311.08396/) (2023)
- [FunnyNet-W: Multimodal Learning of Funny Moments in Videos in the Wild](https://mlsanity.com/paper/2401.04210/) (2024)
- [TACOS: Temporally-aligned Audio CaptiOnS for Language-Audio Pretraining](https://mlsanity.com/paper/2505.07609/) (2025)
- [Bridging Language Gaps in Audio-Text Retrieval](https://mlsanity.com/paper/2406.07012/) (2024)
- [Fusing Audio and Metadata Embeddings Improves Language-based Audio Retrieval](https://mlsanity.com/paper/2406.15897/) (2024)
- [Enhancing Audio-Language Models through Self-Supervised Post-Training with Text-Audio Pairs](https://mlsanity.com/paper/2408.09269/) (2024)
- [Audio-3DVG: Unified Audio -- Point Cloud Fusion for 3D Visual Grounding](https://mlsanity.com/paper/2507.00669/) (2025)

## Abstract (arXiv)

Audio-text retrieval based on natural language descriptions is a challenging task. It involves learning cross-modality alignments between long sequences under inadequate data conditions. In this work, we investigate several audio features as well as sequence aggregation methods for better audio-text alignment. Moreover, through a qualitative analysis we observe that semantic mapping is more important than temporal relations in contextual retrieval. Using pre-trained audio features and a descriptor-based aggregation method, we build our contextual audio-text retrieval system. Specifically, we utilize PANNs features pre-trained on a large sound event dataset and NetRVLAD pooling, which directly works with averaged descriptors. Experiments are conducted on the AudioCaps and CLOTHO datasets, and results are compared with the previous state-of-the-art system. With our proposed system, a significant improvement has been achieved on bidirectional audio-text retrieval, on all metrics including recall, median and mean rank.