# Automated Audio Captioning with Epochal Difficult Captions for Curriculum Learning

arXiv: https://arxiv.org/abs/2206.01918 · HTML: https://mlsanity.com/paper/2206.01918/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 13 of its 21 references that are in the map.
- First posted on arXiv 4 Jun 2022.

## Builds on (most cited first, 13 of 13)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [CIDEr: Consensus-based Image Description Evaluation](https://mlsanity.com/paper/1411.5726/) (2014)
- [SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition](https://mlsanity.com/paper/1904.08779/) (2019)
- [SPICE: Semantic Propositional Image Caption Evaluation](https://mlsanity.com/paper/1607.08822/) (2016)
- [Clotho: An Audio Captioning Dataset](https://mlsanity.com/paper/1910.09387/) (2019)
- [Audio Captioning Transformer](https://mlsanity.com/paper/2107.09817/) (2021)
- [Audio Captioning using Pre-Trained Large-Scale Language Model Guided by Audio-based Similar Caption Retrieval](https://mlsanity.com/paper/2012.07331/) (2020)
- [Investigating Local and Global Information for Automated Audio Captioning with Transfer Learning](https://mlsanity.com/paper/2102.11457/) (2021)
- [A Transformer-based Audio Captioning Model with Keyword Estimation](https://mlsanity.com/paper/2007.00222/) (2020)
- [An Encoder-Decoder Based Audio Captioning System With Transfer and Reinforcement Learning](https://mlsanity.com/paper/2108.02752/) (2021)
- [Automated Audio Captioning using Transfer Learning and Reconstruction Latent Space Similarity Regularization](https://mlsanity.com/paper/2108.04692/) (2021)
- [Improving the Performance of Automated Audio Captioning via Integrating the Acoustic and Semantic Information](https://mlsanity.com/paper/2110.06100/) (2021)

## Abstract (arXiv)

In this paper, we propose an algorithm, Epochal Difficult Captions, to supplement the training of any model for the Automated Audio Captioning task. Epochal Difficult Captions is an elegant evolution to the keyword estimation task that previous work have used to train the encoder of the AAC model. Epochal Difficult Captions modifies the target captions based on a curriculum and a difficulty level determined as a function of current epoch. Epochal Difficult Captions can be used with any model architecture and is a lightweight function that does not increase training time. We test our results on three systems and show that using Epochal Difficult Captions consistently improves performance