mlsanityData to 7 Oct 2026

EnCLAP++: Analyzing the EnCLAP Framework for Optimizing Automated Audio Captioning Performance

arXiv 2409.01201 ↗First posted 2 Sep 2024eess.AS cs.AI cs.SDcited by 3

Not on the map. Why?

Abstract (arXiv)

In this work, we aim to analyze and optimize the EnCLAP framework, a state-of-the-art model in automated audio captioning. We investigate the impact of modifying the acoustic encoder components, explore pretraining with different dataset scales, and study the effectiveness of a reranking scheme. Through extensive experimentation and quantitative analysis of generated captions, we develop EnCLAP++, an enhanced version that significantly surpasses the original.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 23 of its 33 references that are in the map. First posted on arXiv 2 Sep 2024.

Builds on

23 of 33 references.

References · 23

  1. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  2. 2021SoundStream: An End-to-End Neural Audio Codeccited by 756
  3. 2023Neural Codec Language Models are Zero-Shot Text to Speech Synthesizerscited by 737
  4. 2022Large-scale Contrastive Language-Audio Pretraining with Feature Fusion and Keyword-to-Caption Augmentationcited by 707
  5. 2023Simple and Controllable Music Generationcited by 480
  6. 2023High-Fidelity Audio Compression with Improved RVQGANcited by 423
  7. 2019Clotho: An Audio Captioning Datasetcited by 354
  8. 2022AudioGen: Textually Guided Audio Generationcited by 292
  9. 2023WavCaps: A ChatGPT-Assisted Weakly-Labelled Audio Captioning Dataset for Audio-Language Multimodal Researchcited by 266
  10. 2023Pengi: An Audio Language Model for Audio Taskscited by 217
  11. 2021Audio Captioning Transformercited by 53
  12. 2017Automated Audio Captioning with Recurrent Neural Networkscited by 40
11 more
  1. 2021Can Audio Captions Be Evaluated with Image Caption Metrics?cited by 33
  2. 2024EnCLAP: Combining Neural Audio Codec and Audio-Text Joint Embedding for Automated Audio Captioningcited by 28
  3. 2023Prefix tuning for automated audio captioningcited by 26
  4. 2024Codec-SUPERB: An In-Depth Analysis of Sound Codec Modelscited by 25
  5. 2020A Transformer-based Audio Captioning Model with Keyword Estimationcited by 24
  6. 2023Improving Audio Captioning Models with Fine-grained Audio Features, Text Embedding Supervision, and LLM Mix-up Augmentationcited by 19
  7. 2023Adapting a ConvNeXt model to audio classification on AudioSetcited by 13
  8. 2023CoNeTTE: An efficient Audio Captioning system leveraging multiple datasets with Task Embeddingcited by 13
  9. 2022Exploring Train and Test-Time Augmentations for Audio-Language Learningcited by 11
  10. 2024Revisiting Deep Audio-Text Retrieval Through the Lens of Transportationcited by 10
  11. 2023Multitask learning in Audio Captioning: a sentence embedding regression loss acts as a regularizercited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 1.

3 papers.

  1. 2025Reinforcement Learning Outperforms Supervised Fine-Tuning: A Case Study on Audio Question Answeringcited by 64
  2. 2025CLAP-ART: Automated Audio Captioning with Semantic-rich Audio Representation Tokenizercited by 3
  3. 2026Acoustic Prompting via Stage-wise Modulation for Few-Shot Learning in Audio Language Modelscited by 0