# ATRI: Mitigating Multilingual Audio Text Retrieval Inconsistencies by Reducing Data Distribution Errors

arXiv: https://arxiv.org/abs/2502.14627 · HTML: https://mlsanity.com/paper/2502.14627/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 34 references that are in the map.
- First posted on arXiv 20 Feb 2025.

## Builds on (most cited first, 17 of 17)

- [Large-scale Contrastive Language-Audio Pretraining with Feature Fusion and Keyword-to-Caption Augmentation](https://mlsanity.com/paper/2211.06687/) (2022)
- [Language-agnostic BERT Sentence Embedding](https://mlsanity.com/paper/2007.01852/) (2020)
- [Mind the Gap: Understanding the Modality Gap in Multi-modal Contrastive Representation Learning](https://mlsanity.com/paper/2203.02053/) (2022)
- [Clotho: An Audio Captioning Dataset](https://mlsanity.com/paper/1910.09387/) (2019)
- [AudioCLIP: Extending CLIP to Image, Text and Audio](https://mlsanity.com/paper/2106.13043/) (2021)
- [Wav2CLIP: Learning Robust Audio Representations From CLIP](https://mlsanity.com/paper/2110.11499/) (2021)
- [SONAR: Sentence-Level Multimodal and Language-Agnostic Representations](https://mlsanity.com/paper/2308.11466/) (2023)
- [CompA: Addressing the Gap in Compositional Reasoning in Audio-Language Models](https://mlsanity.com/paper/2310.08753/) (2023)
- [CED: Consistent ensemble distillation for audio tagging](https://mlsanity.com/paper/2308.11957/) (2023)
- [ReCLAP: Improving Zero Shot Audio Classification by Describing Sounds](https://mlsanity.com/paper/2409.09213/) (2024)
- [DP-SGD Without Clipping: The Lipschitz Neural Network Way](https://mlsanity.com/paper/2305.16202/) (2023)
- [Do we really have to filter out random noise in pre-training data for language models?](https://mlsanity.com/paper/2502.06604/) (2025)
- [Imbalanced Open Set Domain Adaptation via Moving-threshold Estimation and Gradual Alignment](https://mlsanity.com/paper/2303.04393/) (2023)
- [Bridging Language Gaps in Audio-Text Retrieval](https://mlsanity.com/paper/2406.07012/) (2024)
- [Multilingual Audio Captioning using machine translated data](https://mlsanity.com/paper/2309.07615/) (2023)
- [DiffATR: Diffusion-based Generative Modeling for Audio-Text Retrieval](https://mlsanity.com/paper/2409.10025/) (2024)
- [Improving the Consistency in Cross-Lingual Cross-Modal Retrieval with 1-to-K Contrastive Learning](https://mlsanity.com/paper/2406.18254/) (2024)

## Built on by (most cited first, 6 of 6)

- [SupCLAP: Controlling Optimization Trajectory Drift in Audio-Text Contrastive Learning with Support Vector Regularization](https://mlsanity.com/paper/2509.21033/) (2025)
- [DermoGPT: Open Weights and Open Data for Morphology-Grounded Dermatological Reasoning MLLMs](https://mlsanity.com/paper/2601.01868/) (2026)
- [HAD: Hybrid Architecture Distillation Outperforms Teacher in Genomic Sequence Modeling](https://mlsanity.com/paper/2505.20836/) (2025)
- [Multiscale Adaptive Conflict-Balancing Model For Multimedia Deepfake Detection](https://mlsanity.com/paper/2505.12966/) (2025)
- [Open Set Domain Adaptation with Vision-language models via Gradient-aware Separation](https://mlsanity.com/paper/2505.13507/) (2025)
- [ASK: Adaptive Self-improving Knowledge Framework for Audio Text Retrieval](https://mlsanity.com/paper/2512.19703/) (2025)

## Abstract (arXiv)

Multilingual audio-text retrieval (ML-ATR) is a challenging task that aims to retrieve audio clips or multilingual texts from databases. However, existing ML-ATR schemes suffer from inconsistencies for instance similarity matching across languages. We theoretically analyze the inconsistency in terms of both multilingual modal alignment direction error and weight error, and propose the theoretical weight error upper bound for quantifying the inconsistency. Based on the analysis of the weight error upper bound, we find that the inconsistency problem stems from the data distribution error caused by random sampling of languages. We propose a consistent ML-ATR scheme using 1-to-k contrastive learning and audio-English co-anchor contrastive learning, aiming to mitigate the negative impact of data distribution error on recall and consistency in ML-ATR. Experimental results on the translated AudioCaps and Clotho datasets show that our scheme achieves state-of-the-art performance on recall and consistency metrics for eight mainstream languages, including English. Our code will be available at https://github.com/ATRI-ACL/ATRI-ACL.