# Adaptive Loss Balancing for Multi-Task Bioacoustic Classification of Bird Species and Call Types

arXiv: https://arxiv.org/abs/2607.03304 · HTML: https://mlsanity.com/paper/2607.03304/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 22 of its 50 references that are in the map.
- First posted on arXiv 3 Jul 2026.

## Builds on (most cited first, 22 of 22)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [A ConvNet for the 2020s](https://mlsanity.com/paper/2201.03545/) (2022)
- [Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics](https://mlsanity.com/paper/1705.07115/) (2017)
- [Understanding intermediate layers using linear classifier probes](https://mlsanity.com/paper/1610.01644/) (2016)
- [CvT: Introducing Convolutions to Vision Transformers](https://mlsanity.com/paper/2103.15808/) (2021)
- [A Survey on Multi-Task Learning](https://mlsanity.com/paper/1707.08114/) (2017)
- [GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networks](https://mlsanity.com/paper/1711.02257/) (2017)
- [data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Language](https://mlsanity.com/paper/2202.03555/) (2022)
- [AST: Audio Spectrogram Transformer](https://mlsanity.com/paper/2104.01778/) (2021)
- [End-to-End Multi-Task Learning with Attention](https://mlsanity.com/paper/1803.10704/) (2018)
- [BEATs: Audio Pre-Training with Acoustic Tokenizers](https://mlsanity.com/paper/2212.09058/) (2022)
- [Masked Autoencoders that Listen](https://mlsanity.com/paper/2207.06405/) (2022)
- [Seeing biodiversity: perspectives in machine learning for wildlife conservation](https://mlsanity.com/paper/2110.12951/) (2021)
- [Scalable Pre-training of Large Autoregressive Image Models](https://mlsanity.com/paper/2401.08541/) (2024)
- [Computational bioacoustics with deep learning: a review and roadmap](https://mlsanity.com/paper/2112.06725/) (2021)
- [AVES: Animal Vocalization Encoder based on Self-Supervision](https://mlsanity.com/paper/2210.14493/) (2022)
- [Global birdsong embeddings enable superior transfer learning for bioacoustic classification](https://mlsanity.com/paper/2307.06292/) (2023)
- [BirdSet: A Large-Scale Dataset for Audio Classification in Avian Bioacoustics](https://mlsanity.com/paper/2403.10380/) (2024)
- [Can Masked Autoencoders Also Listen to Birds?](https://mlsanity.com/paper/2504.12880/) (2025)
- [NIPS4Bplus: a richly annotated birdsong audio dataset](https://mlsanity.com/paper/1811.02275/) (2018)
- [Foundation Models for Bioacoustics -- a Comparative Review](https://mlsanity.com/paper/2508.01277/) (2025)
- [Domain-Invariant Representation Learning of Bird Sounds](https://mlsanity.com/paper/2409.08589/) (2024)

## Abstract (arXiv)

Reliable analysis of bird vocalisations in passive acoustic monitoring requires models handling multiple, imbalanced annotation targets. We extend BirdCallNet for joint species and call-type classification on the long-tailed WiWa dataset and investigate how task-loss balancing interacts with pretrained representations and adaptation depth. We evaluate four bird-domain encoders, ConvNeXtBS, EAT, BirdMAE, and ProtoCLR, with separate species and call-type heads under linear probing, attentive probing, and full fine-tuning. A manually tuned fixed objective is compared with homoscedastic uncertainty weighting and Dynamic Weight Averaging across all three adaptation regimes, while GradNorm is evaluated only under full fine-tuning. Results indicate that the factorised multi-task formulation yields the most consistent improvements over the combined single-task baseline for call-type recognition, while its effect on species recognition depends on the adaptation regime. Full fine-tuning is not consistently optimal: ConvNeXtBS achieves the highest mean species performance under linear probing, whereas BirdMAE provides the strongest call-type performance under attentive probing. Adaptive weighting benefits species recognition more consistently than call-type recognition. Uncertainty weighting is particularly effective for species recognition under attentive probing, whereas Dynamic Weight Averaging is generally stronger for the same task under full fine-tuning. GradNorm achieves competitive call-type performance for selected backbones but consistently underperforms other weighting strategies for species recognition and incurs higher computational and memory costs. Overall, the preferred loss-balancing strategy depends on the backbone, adaptation regime, and target task, while frozen-backbone adaptation can provide a more favourable performance-efficiency trade-off than end-to-end fine-tuning.