mlsanityData to 14 Oct 2026

TimbreCLIP: Connecting Timbre to Text and Images

arXiv 2211.11225 ↗First posted 21 Nov 2022cs.SD cs.LG eess.AScited by 5

Not on the map. Why?

Abstract (arXiv)

We present work in progress on TimbreCLIP, an audio-text cross modal embedding trained on single instrument notes. We evaluate the models with a cross-modal retrieval task on synth patches. Finally, we demonstrate the application of TimbreCLIP on two tasks: text-driven audio equalization and timbre to image generation.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 13 of its 15 references that are in the map. First posted on arXiv 21 Nov 2022.

Builds on

13 of 15 references.

References · 13

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  3. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  4. 2022Large-scale Contrastive Language-Audio Pretraining with Feature Fusion and Keyword-to-Caption Augmentationcited by 719
  5. 2021AudioCLIP: Extending CLIP to Image, Text and Audiocited by 339
  6. 2022VQGAN-CLIP: Open Domain Image Generation and Editing with Natural Language Guidancecited by 285
  7. 2017Neural Audio Synthesis of Musical Notes with WaveNet Autoencoderscited by 225
  8. 2021Wav2CLIP: Learning Robust Audio Representations From CLIPcited by 213
  9. 2022MuLan: A Joint Embedding of Music Audio and Natural Languagecited by 87
  10. 2022Contrastive Audio-Language Learning for Musiccited by 33
  11. 2022SpeechCLIP: Integrating Speech with Pre-Trained Vision and Language Modelcited by 29
  12. 2021One Billion Audio Sounds from GPU-enabled Modular Synthesiscited by 11
1 more
  1. 2022Robust Sound-Guided Image Manipulationcited by 7

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 2; 2024: 1; 2025: 1.

5 papers.

  1. 2023Soundini: Sound-Guided Diffusion for Natural Video Editingcited by 14
  2. 2023SynthScribe: Deep Multimodal Tools for Synthesizer Sound Retrieval and Explorationcited by 3
  3. 2024Contrastive Learning from Synthetic Audio Doppelg"angerscited by 1
  4. 2022Audio Latent Space Cartographycited by 0
  5. 2025Testing chatbots on the creation of encoders for audio conditioned image generationcited by 0