mlsanityData to 14 Oct 2026

Language-Based Audio Retrieval with Converging Tied Layers and Contrastive Loss

arXiv 2206.14659 ↗First posted 29 Jun 2022cs.SD cs.CL cs.IRcited by 0

Not on the map. Why?

Abstract (arXiv)

In this paper, we tackle the new Language-Based Audio Retrieval task proposed in DCASE 2022. Firstly, we introduce a simple, scalable architecture which ties both the audio and text encoder together. Secondly, we show that using this architecture along with contrastive loss allows the model to significantly beat the performance of the baseline model. Finally, in addition to having an extremely low training memory requirement, we are able to use pretrained models as it is without needing to finetune them. We test our methods and show that using a combination of our methods beats the baseline scores significantly.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 16 of its 19 references that are in the map. First posted on arXiv 29 Jun 2022.

Builds on

16 of 19 references.

References · 16

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  4. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,899
  5. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,375
  6. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,148
  7. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
  8. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,474
  9. 2017RACE: Large-scale ReAding Comprehension Dataset From Examinationscited by 861
  10. 2020Contrastive Learning of Medical Visual Representations from Paired Images and Textcited by 641
  11. 2019PANNs: Large-Scale Pretrained Audio Neural Networks for Audio Pattern Recognitioncited by 574
  12. 2019Clotho: An Audio Captioning Datasetcited by 358
4 more
  1. 2021AudioCLIP: Extending CLIP to Image, Text and Audiocited by 339
  2. 2021Text-to-Audio Grounding: Building Correspondence Between Captions and Sound Eventscited by 16
  3. 2021Automated Audio Captioning using Transfer Learning and Reconstruction Latent Space Similarity Regularizationcited by 12
  4. 2021Unsupervised Audio-Caption Aligning Learns Correspondences between Individual Sound Events and Textual Phrasescited by 8

Built on by

Not cited yet.