# VI-PANN: Harnessing Transfer Learning and Uncertainty-Aware Variational Inference for Improved Generalization in Audio Pattern Recognition

arXiv: https://arxiv.org/abs/2401.05531 · HTML: https://mlsanity.com/paper/2401.05531/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 38 references that are in the map.
- First posted on arXiv 10 Jan 2024.

## Builds on (most cited first, 15 of 15)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?](https://mlsanity.com/paper/1703.04977/) (2017)
- [PANNs: Large-Scale Pretrained Audio Neural Networks for Audio Pattern Recognition](https://mlsanity.com/paper/1912.10211/) (2019)
- [Variational Dropout and the Local Reparameterization Trick](https://mlsanity.com/paper/1506.02557/) (2015)
- [Concrete Dropout](https://mlsanity.com/paper/1705.07832/) (2017)
- [Flipout: Efficient Pseudo-Independent Weight Perturbations on Mini-Batches](https://mlsanity.com/paper/1803.04386/) (2018)
- [A Systematic Comparison of Bayesian Deep Learning Robustness in Diabetic Retinopathy Tasks](https://mlsanity.com/paper/1912.10481/) (2019)
- [OmniVec: Learning robust representations with cross modal sharing](https://mlsanity.com/paper/2311.05709/) (2023)
- [Transfer learning for music classification and regression tasks](https://mlsanity.com/paper/1703.09179/) (2017)
- [Specifying Weight Priors in Bayesian Deep Neural Networks with Empirical Bayes](https://mlsanity.com/paper/1906.05323/) (2019)
- [ERANNs: Efficient Residual Audio Neural Networks for Audio Pattern Recognition](https://mlsanity.com/paper/2106.01621/) (2021)
- [Distributed Weight Consolidation: A Brain Segmentation Case Study](https://mlsanity.com/paper/1805.10863/) (2018)
- [Towards Dependable Autonomous Systems Based on Bayesian Deep Learning Components](https://mlsanity.com/paper/2301.05297/) (2023)

## Built on by (most cited first, 1 of 1)

- [MAGIC-Enhanced Keyword Prompting for Zero-Shot Audio Captioning with CLIP Models](https://mlsanity.com/paper/2509.12591/) (2025)

## Abstract (arXiv)

Transfer learning (TL) is an increasingly popular approach to training deep learning (DL) models that leverages the knowledge gained by training a foundation model on diverse, large-scale datasets for use on downstream tasks where less domain- or task-specific data is available. The literature is rich with TL techniques and applications; however, the bulk of the research makes use of deterministic DL models which are often uncalibrated and lack the ability to communicate a measure of epistemic (model) uncertainty in prediction. Unlike their deterministic counterparts, Bayesian DL (BDL) models are often well-calibrated, provide access to epistemic uncertainty for a prediction, and are capable of achieving competitive predictive performance. In this study, we propose variational inference pre-trained audio neural networks (VI-PANNs). VI-PANNs are a variational inference variant of the popular ResNet-54 architecture which are pre-trained on AudioSet, a large-scale audio event detection dataset. We evaluate the quality of the resulting uncertainty when transferring knowledge from VI-PANNs to other downstream acoustic classification tasks using the ESC-50, UrbanSound8K, and DCASE2013 datasets. We demonstrate, for the first time, that it is possible to transfer calibrated uncertainty information along with knowledge from upstream tasks to enhance a model's capability to perform downstream tasks.