# On Batching Variable Size Inputs for Training End-to-End Speech Enhancement Systems

arXiv: https://arxiv.org/abs/2301.10587 · HTML: https://mlsanity.com/paper/2301.10587/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 39 references that are in the map.
- First posted on arXiv 25 Jan 2023.

## Builds on (most cited first, 13 of 13)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima](https://mlsanity.com/paper/1609.04836/) (2016)
- [Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictions](https://mlsanity.com/paper/1712.05884/) (2017)
- [SDR - half-baked or well done?](https://mlsanity.com/paper/1811.02508/) (2018)
- [Conv-TasNet: Surpassing Ideal Time-Frequency Magnitude Masking for Speech Separation](https://mlsanity.com/paper/1809.07454/) (2018)
- [Supervised Speech Separation Based on Deep Learning: An Overview](https://mlsanity.com/paper/1708.07524/) (2017)
- [Convergence Analysis of Distributed Stochastic Gradient Descent with Shuffling](https://mlsanity.com/paper/1709.10432/) (2017)
- [DCCRN+: Channel-wise Subband DCCRN with SNR Estimation for Speech Enhancement](https://mlsanity.com/paper/2106.08672/) (2021)
- [Exploring the Best Loss Function for DNN-Based Low-latency Speech Enhancement with Temporal Convolutional Networks](https://mlsanity.com/paper/2005.11611/) (2020)
- [An Empirical Study of Mini-Batch Creation Strategies for Neural Machine Translation](https://mlsanity.com/paper/1706.05765/) (2017)
- [TPARN: Triple-path Attentive Recurrent Network for Time-domain Multichannel Speech Enhancement](https://mlsanity.com/paper/2110.10757/) (2021)
- [Accelerating recurrent neural network training using sequence bucketing and multi-GPU data parallelization](https://mlsanity.com/paper/1708.05604/) (2017)
- [A comprehensive study of batch construction strategies for recurrent neural networks in MXNet](https://mlsanity.com/paper/1705.02414/) (2017)

## Built on by (most cited first, 5 of 5)

- [Dataset Decomposition: Faster LLM Training with Variable Sequence Length Curriculum](https://mlsanity.com/paper/2405.13226/) (2024)
- [Investigating the Design Space of Diffusion Models for Speech Enhancement](https://mlsanity.com/paper/2312.04370/) (2023)
- [Diffusion-Based Speech Enhancement in Matched and Mismatched Conditions Using a Heun-Based Sampler](https://mlsanity.com/paper/2312.02683/) (2023)
- [Assessing the Generalization Gap of Learning-Based Speech Enhancement Systems in Noisy and Reverberant Environments](https://mlsanity.com/paper/2309.06183/) (2023)
- [Noise-Robust Target-Speaker Voice Activity Detection Through Self-Supervised Pretraining](https://mlsanity.com/paper/2501.03184/) (2025)

## Abstract (arXiv)

The performance of neural network-based speech enhancement systems is primarily influenced by the model architecture, whereas training times and computational resource utilization are primarily affected by training parameters such as the batch size. Since noisy and reverberant speech mixtures can have different duration, a batching strategy is required to handle variable size inputs during training, in particular for state-of-the-art end-to-end systems. Such strategies usually strive for a compromise between zero-padding and data randomization, and can be combined with a dynamic batch size for a more consistent amount of data in each batch. However, the effect of these strategies on resource utilization and more importantly network performance is not well documented. This paper systematically investigates the effect of different batching strategies and batch sizes on the training statistics and speech enhancement performance of a Conv-TasNet, evaluated in both matched and mismatched conditions. We find that using a small batch size during training improves performance in both conditions for all batching strategies. Moreover, using sorted or bucket batching with a dynamic batch size allows for reduced training time and GPU memory usage while achieving similar performance compared to random batching with a fixed batch size.