# A comprehensive study of batch construction strategies for recurrent neural networks in MXNet

arXiv: https://arxiv.org/abs/1705.02414 · HTML: https://mlsanity.com/paper/1705.02414/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 May 2017.

## Built on by (most cited first, 2 of 2)

- [Training and Inference Efficiency of Encoder-Decoder Speech Models](https://mlsanity.com/paper/2503.05931/) (2025)
- [On Batching Variable Size Inputs for Training End-to-End Speech Enhancement Systems](https://mlsanity.com/paper/2301.10587/) (2023)

## Abstract (arXiv)

In this work we compare different batch construction methods for mini-batch training of recurrent neural networks. While popular implementations like TensorFlow and MXNet suggest a bucketing approach to improve the parallelization capabilities of the recurrent training process, we propose a simple ordering strategy that arranges the training sequences in a stochastic alternatingly sorted way. We compare our method to sequence bucketing as well as various other batch construction strategies on the CHiME-4 noisy speech recognition corpus. The experiments show that our alternated sorting approach is able to compete both in training time and recognition performance while being conceptually simpler to implement.