# Mini-batch Serialization: CNN Training with Inter-layer Data Reuse

arXiv: https://arxiv.org/abs/1810.00307 · HTML: https://mlsanity.com/paper/1810.00307/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Sep 2018.

## Built on by (most cited first, 8 of 8)

- [Rethinking "Batch" in BatchNorm](https://mlsanity.com/paper/2105.07576/) (2021)
- [GradPIM: A Practical Processing-in-DRAM Architecture for Gradient Descent](https://mlsanity.com/paper/2102.07511/) (2021)
- [SGCN: Exploiting Compressed-Sparse Features in Deep Graph Convolutional Network Accelerators](https://mlsanity.com/paper/2301.10388/) (2023)
- [BlockLLM: Multi-tenant Finer-grained Serving for Large Language Models](https://mlsanity.com/paper/2404.18322/) (2024)
- [Towards Memory-Efficient Neural Networks via Multi-Level in situ Generation](https://mlsanity.com/paper/2108.11430/) (2021)
- [Optimizer Fusion: Efficient Training with Better Locality and Parallelism](https://mlsanity.com/paper/2104.00237/) (2021)
- [EcoFlow: Efficient Convolutional Dataflows for Low-Power Neural Network Accelerators](https://mlsanity.com/paper/2202.02310/) (2022)
- [Slice-and-Forge: Making Better Use of Caches for Graph Convolutional Network Accelerators](https://mlsanity.com/paper/2301.09813/) (2023)

## Abstract (arXiv)

Training convolutional neural networks (CNNs) requires intense computations and high memory bandwidth. We find that bandwidth today is over-provisioned because most memory accesses in CNN training can be eliminated by rearranging computation to better utilize on-chip buffers and avoid traffic resulting from large per-layer memory footprints. We introduce the MBS CNN training approach that significantly reduces memory traffic by partially serializing mini-batch processing across groups of layers. This optimizes reuse within on-chip buffers and balances both intra-layer and inter-layer reuse. We also introduce the WaveCore CNN training accelerator that effectively trains CNNs in the MBS approach with high functional-unit utilization. Combined, WaveCore and MBS reduce DRAM traffic by 75%, improve performance by 53%, and save 26% system energy for modern deep CNN training compared to conventional training mechanisms and accelerators.