# Maya: Optimizing Deep Learning Training Workloads using GPU Runtime Emulation

arXiv: https://arxiv.org/abs/2503.20191 · HTML: https://mlsanity.com/paper/2503.20191/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 46 references that are in the map.
- First posted on arXiv 26 Mar 2025.

## Builds on (most cited first, 21 of 21)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism](https://mlsanity.com/paper/1909.08053/) (2019)
- [Tune: A Research Platform for Distributed Model Selection and Training](https://mlsanity.com/paper/1807.05118/) (2018)
- [The CMA Evolution Strategy: A Tutorial](https://mlsanity.com/paper/1604.00772/) (2016)
- [Reducing Activation Recomputation in Large Transformer Models](https://mlsanity.com/paper/2205.05198/) (2022)
- [Beyond Data and Model Parallelism for Deep Neural Networks](https://mlsanity.com/paper/1807.05358/) (2018)
- [Vidur: A Large-Scale Simulation Framework For LLM Inference](https://mlsanity.com/paper/2405.05465/) (2024)
- [A Hybrid Tensor-Expert-Data Parallelism Approach to Optimize Mixture-of-Experts Training](https://mlsanity.com/paper/2303.06318/) (2023)
- [ASTRA-sim2.0: Modeling Hierarchical Networks and Disaggregated Systems for Large-model Training at Scale](https://mlsanity.com/paper/2303.14006/) (2023)
- [A Runtime-Based Computational Performance Predictor for Deep Neural Network Training](https://mlsanity.com/paper/2102.00527/) (2021)
- [vTrain: A Simulation Framework for Evaluating Cost-effective and Compute-optimal Large Language Model Training](https://mlsanity.com/paper/2312.12391/) (2023)
- [Daydream: Accurately Estimating the Efficacy of Optimizations for DNN Training](https://mlsanity.com/paper/2006.03318/) (2020)
- [Memory Optimization for Deep Networks](https://mlsanity.com/paper/2010.14501/) (2020)
- [Forecasting GPU Performance for Deep Learning Training and Inference](https://mlsanity.com/paper/2407.13853/) (2024)
- [APEX: An Extensible and Dynamism-Aware Simulator for Automated Parallel Execution in LLM Serving](https://mlsanity.com/paper/2411.17651/) (2024)
- [Singularity: Planet-Scale, Preemptive and Elastic Scheduling of AI Workloads](https://mlsanity.com/paper/2202.07848/) (2022)
- [Proteus: Simulating the Performance of Distributed DNN Training](https://mlsanity.com/paper/2306.02267/) (2023)
- [A Simple Model for Portable and Fast Prediction of Execution Time and Power Consumption of GPU Kernels](https://mlsanity.com/paper/2001.07104/) (2020)
- [DistSim: A performance model of large-scale hybrid distributed DNN training](https://mlsanity.com/paper/2306.08423/) (2023)

## Built on by (most cited first, 1 of 1)

- [ASTRA-sim 3.0: Next-Level Distributed Machine Learning Simulations via High-Fidelity GPU and Infrastructure Modeling](https://mlsanity.com/paper/2606.10440/) (2026)

## Abstract (arXiv)

Training large foundation models costs hundreds of millions of dollars, making deployment optimization critical. Current approaches require machine learning engineers to manually craft training recipes through error-prone trial-and-error on expensive compute clusters. To enable efficient exploration of training configurations, researchers have developed performance modeling systems. However, these systems force users to translate their workloads into custom specification languages, introducing a fundamental semantic gap between the actual workload and its representation. This gap creates an inherent tradeoff: systems must either support a narrow set of workloads to maintain usability, require complex specifications that limit practical adoption, or compromise prediction accuracy with simplified performance models. We present Maya, a performance modeling system that eliminates these tradeoffs through transparent device emulation. By operating at the narrow interface between training frameworks and accelerator devices, Maya can capture complete workload behavior without requiring code modifications or translations. Maya intercepts device API calls from unmodified training code to directly observe low-level operations, enabling accurate performance prediction while maintaining both ease of use and generality. Our evaluation shows Maya achieves less than 5% prediction error across diverse models and optimization strategies, identifying configurations that reduce training costs by up to 56% compared to existing approaches.