# An Analysis of Collocation on GPUs for Deep Learning Training

arXiv: https://arxiv.org/abs/2209.06018 · HTML: https://mlsanity.com/paper/2209.06018/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 44 references that are in the map.
- First posted on arXiv 13 Sep 2022.

## Builds on (most cited first, 15 of 15)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Identity Mappings in Deep Residual Networks](https://mlsanity.com/paper/1603.05027/) (2016)
- [EfficientNetV2: Smaller Models and Faster Training](https://mlsanity.com/paper/2104.00298/) (2021)
- [Going deeper with Image Transformers](https://mlsanity.com/paper/2103.17239/) (2021)
- [Image Segmentation Using Deep Learning: A Survey](https://mlsanity.com/paper/2001.05566/) (2020)
- [A Downsampled Variant of ImageNet as an Alternative to the CIFAR datasets](https://mlsanity.com/paper/1707.08819/) (2017)
- [Deep Learning Recommendation Model for Personalization and Recommendation Systems](https://mlsanity.com/paper/1906.00091/) (2019)
- [Analysis of Large-Scale Multi-Tenant GPU Clusters for DNN Training Workloads](https://mlsanity.com/paper/1901.05758/) (2019)
- [A survey on Semi-, Self- and Unsupervised Learning for Image Classification](https://mlsanity.com/paper/2002.08721/) (2020)
- [MISO: Exploiting Multi-Instance GPU Capability on Multi-Tenant Systems for Machine Learning](https://mlsanity.com/paper/2207.11428/) (2022)
- [CROSSBOW: Scaling Deep Learning with Small Batch Sizes on Multi-GPU Servers](https://mlsanity.com/paper/1901.02244/) (2019)
- [Horizontally Fused Training Array: An Effective Hardware Utilization Squeezer for Training Novel Deep Learning Models](https://mlsanity.com/paper/2102.02344/) (2021)
- [Training for Speech Recognition on Coprocessors](https://mlsanity.com/paper/2003.12366/) (2020)

## Built on by (most cited first, 6 of 6)

- [Contrastive Forward-Forward: A Training Algorithm of Vision Transformer](https://mlsanity.com/paper/2502.00571/) (2025)
- [TensorSocket: Shared Data Loading for Deep Learning Training](https://mlsanity.com/paper/2409.18749/) (2024)
- [Position: Neglecting the Sustainability of AI is Fuelling a Global AI Arms Race](https://mlsanity.com/paper/2502.20016/) (2025)
- [PREBA: A Hardware/Software Co-Design for Multi-Instance GPU based AI Inference Servers](https://mlsanity.com/paper/2411.19114/) (2024)
- [AEGIS: Runtime-Guided GPU Collocation for Multi-Tenant Deep Learning Training](https://mlsanity.com/paper/2508.19073/) (2025)
- [AIBuildAI-2.5: Efficient Autonomous AI Model Development Through LLM-Guided Tree Search](https://mlsanity.com/paper/2609.25047/) (2026)

## Abstract (arXiv)

Deep learning training is an expensive process that extensively uses GPUs, but not all model training saturates modern powerful GPUs. Multi-Instance GPU (MIG) is a new technology introduced by NVIDIA that can partition a GPU to better-fit workloads that do not require all the memory and compute resources of a full GPU. In this paper, we examine the performance of a MIG-enabled A100 GPU under deep learning workloads containing various sizes and combinations of models. We contrast the benefits of MIG to older workload collocation methods on GPUs: na"ively submitting multiple processes on the same GPU and utilizing Multi-Process Service (MPS). Our results demonstrate that collocating multiple model training runs may yield significant benefits. In certain cases, it can lead up to four times training throughput despite increased epoch time. On the other hand, the aggregate memory footprint and compute needs of the models trained in parallel must fit the available memory and compute resources of the GPU. MIG can be beneficial thanks to its interference-free partitioning, especially when the sizes of the models align with the MIG partitioning options. MIG's rigid partitioning, however, may create sub-optimal GPU utilization for more dynamic mixed workloads. In general, we recommend MPS as the best performing and most flexible form of collocation for model training for a single user submitting training jobs.