mlsanityData to 14 Oct 2026

Probing Low-Level Acoustic Attribute Encoding in CLAP Audio Embeddings

arXiv 2607.03806 ↗First posted 4 Jul 2026eess.AS cs.AI eess.SPcited by 1

Not on the map. Why?

Abstract (arXiv)

Audio foundation models are widely adopted as general-purpose feature extractors, yet the internal structure of their learned representations remains insufficiently understood. In this work, we analyze CLAP audio embeddings through a probing framework, studying the encoding of three fundamental perceptual dimensions: reverberation (RT60), loudness (LUFS), and spectral content, measured via spectral centroid (SC) and relative pitch (RP). Probes of increasing complexity are trained to predict each attribute from frozen embeddings across five datasets spanning noise, speech, monophonic musical notes, and music mixtures. Our primary finding is that all of these attributes are reliably recoverable from the CLAP embedding space across the examined datasets. Within this global picture, two encoding regimes emerge: RT60, LUFS, and RP are approximately linearly encoded, while SC requires non-linear probes. Both regimes generalize across eight additional audio foundation models, with the notable exception that amplitude-invariant architectures discard loudness entirely by construction. The identified linear feature directions are geometrically consistent across datasets for RT60 and LUFS, while highly domain-specific for RP. Finally, we provide a qualitative demonstration of cross-modal consistency, showing that text embeddings of acoustic descriptors align geometrically with the identified RT60 feature direction.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 29 of its 38 references that are in the map. First posted on arXiv 4 Jul 2026.

Builds on

29 of 38 references.

References · 29

  1. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
  2. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
  3. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,657
  4. 2022Large-scale Contrastive Language-Audio Pretraining with Feature Fusion and Keyword-to-Caption Augmentationcited by 719
  5. 2016CNN Architectures for Large-Scale Audio Classificationcited by 704
  6. 2023Make-An-Audio: Text-To-Audio Generation with Prompt-Enhanced Diffusion Modelscited by 324
  7. 2017Neural Audio Synthesis of Musical Notes with WaveNet Autoencoderscited by 225
  8. 2023MERT: Acoustic Music Understanding Model with Large-Scale Self-supervised Trainingcited by 181
  9. 2022HTS-AT: A Hierarchical Token-Semantic Audio Transformer for Sound Classification and Detectioncited by 176
  10. 2023Adapting Frechet Audio Distance for Generative Music Evaluationcited by 92
  11. 2023Natural Language Supervision for General-Purpose Audio Representationscited by 61
  12. 2018gpuRIR: A Python Library for Room Impulse Response Simulation with GPU Accelerationcited by 46
17 more
  1. 2024PAM: Prompting Audio-Language Models for Audio Quality Assessmentcited by 20
  2. 2019Probing the Information Encoded in X-vectorscited by 18
  3. 2024T-CLAP: Temporal-Enhanced Contrastive Language-Audio Pretrainingcited by 18
  4. 2025AudioLens: A Closer Look at Auditory Attribute Perception of Large Audio-Language Modelscited by 18
  5. 2024DRCap: Decoding CLAP Latents with Retrieval-Augmented Generation for Zero-shot Audio Captioningcited by 11
  6. 2024Text2FX: Harnessing CLAP Embeddings for Text-Guided Audio Effectscited by 9
  7. 2024AND: Audio Network Dissection for Interpreting Deep Acoustic Modelscited by 7
  8. 2025Fx-Encoder++: Extracting Instrument-Wise Audio Effects Representations from Mixturescited by 6
  9. 2025SonicMaster: Towards Controllable All-in-One Music Restoration and Masteringcited by 6
  10. 2025TokenSynth: A Token-based Neural Synthesizer for Instrument Cloning and Text-to-Instrumentcited by 5
  11. 2025Transformation of audio embeddings into interpretable, concept-based representationscited by 5
  12. 2026Causal Tracing of Audio-Text Fusion in Large Audio Language Modelscited by 5
  13. 2025Investigating the Sensitivity of Pre-trained Audio Embeddings to Common Effectscited by 3
  14. 2025Spatial-CLAP: Learning Spatially-Aware audio--text Embeddings for Multi-Source Conditionscited by 2
  15. 2025FlowSynth: Instrument Generation Through Distributional Flow Matching and Test-Time Searchcited by 2
  16. 2025Listen through the Sound: Generative Speech Restoration Leveraging Acoustic Context Representationcited by 1
  17. 2025Do Joint Language-Audio Embeddings Encode Perceptual Timbre Semantics?cited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2026How Much Audio Is Left In An Embedding? An Inversion Audit Of Audio Encoders
  2. 2025Learning Interpretable Features in Audio Latent Spaces via Sparse Autoencoders
  3. 2025Music2Latent2: Audio Compression with Summary Embeddings and Autoregressive Decoding
  4. 2026Feature Encoding in VAE-based Audio Decoders: Effects of Input, Depth and Distribution
  5. 2025Unmute the Patch Tokens: Rethinking Probing in Multi-Label Audio Classification
  6. 2026SonicBench: Dissecting the Physical Perception Bottleneck in Large Audio Language Models
  7. 2026From Objectives to Applications: Aligning Architectural Biases in Audio Self-Supervised Learning ↗
  8. 2026Discriminating real and synthetic super-resolved audio samples using embedding-based classifiers