# Audio Captioning using Gated Recurrent Units

arXiv: https://arxiv.org/abs/2006.03391 · HTML: https://mlsanity.com/paper/2006.03391/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Jun 2020.

## Built on by (most cited first, 4 of 4)

- [MusCaps: Generating Captions for Music Audio](https://mlsanity.com/paper/2104.11984/) (2021)
- [Automated Audio Captioning using Transfer Learning and Reconstruction Latent Space Similarity Regularization](https://mlsanity.com/paper/2108.04692/) (2021)
- [Evaluating Off-the-Shelf Machine Listening and Natural Language Models for Automated Audio Captioning](https://mlsanity.com/paper/2110.07410/) (2021)
- [Audio Captioning with Composition of Acoustic and Semantic Information](https://mlsanity.com/paper/2105.06355/) (2021)

## Abstract (arXiv)

Audio captioning is a recently proposed task for automatically generating a textual description of a given audio clip. In this study, a novel deep network architecture with audio embeddings is presented to predict audio captions. Within the aim of extracting audio features in addition to log Mel energies, VGGish audio embedding model is used to explore the usability of audio embeddings in the audio captioning task. The proposed architecture encodes audio and text input modalities separately and combines them before the decoding stage. Audio encoding is conducted through Bi-directional Gated Recurrent Unit (BiGRU) while GRU is used for the text encoding phase. Following this, we evaluate our model by means of the newly published audio captioning performance dataset, namely Clotho, to compare the experimental results with the literature. Our experimental results show that the proposed BiGRU-based deep model outperforms the state of the art results.