# Predicting the Computational Cost of Deep Learning Models

arXiv: https://arxiv.org/abs/1811.11880 · HTML: https://mlsanity.com/paper/1811.11880/

## Key facts

- Cited by 49 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Nov 2018.

## Built on by (most cited first, 30 of 49)

- [CODA-Prompt: COntinual Decomposed Attention-based Prompting for Rehearsal-Free Continual Learning](https://mlsanity.com/paper/2211.13218/) (2022)
- [A Closer Look at Rehearsal-Free Continual Learning](https://mlsanity.com/paper/2203.17269/) (2022)
- [Self-Expansion of Pre-trained Models with Mixture of Adapters for Continual Learning](https://mlsanity.com/paper/2403.18886/) (2024)
- [A Runtime-Based Computational Performance Predictor for Deep Neural Network Training](https://mlsanity.com/paper/2102.00527/) (2021)
- [Green Edge AI: A Contemporary Survey](https://mlsanity.com/paper/2312.00333/) (2023)
- [Regular Polytope Networks](https://mlsanity.com/paper/2103.15632/) (2021)
- [FusionAI: Decentralized Training and Deploying LLMs with Massive Consumer-Level GPUs](https://mlsanity.com/paper/2309.01172/) (2023)
- [LLM-Pilot: Characterize and Optimize Performance of your LLM Inference Services](https://mlsanity.com/paper/2410.02425/) (2024)
- [Building a Performance Model for Deep Learning Recommendation Model Training on GPUs](https://mlsanity.com/paper/2201.07821/) (2022)
- [Pretraining Decision Transformers with Reward Prediction for In-Context Multi-task Structured Bandit Learning](https://mlsanity.com/paper/2406.05064/) (2024)
- [Lie-Poisson Neural Networks (LPNets): Data-Based Computing of Hamiltonian Systems with Symmetries](https://mlsanity.com/paper/2308.15349/) (2023)
- [Benchmarking Resource Usage for Efficient Distributed Deep Learning](https://mlsanity.com/paper/2201.12423/) (2022)
- [Continual Adaptation of Vision Transformers for Federated Learning](https://mlsanity.com/paper/2306.09970/) (2023)
- [All-optical image denoising using a diffractive visual processor](https://mlsanity.com/paper/2309.09215/) (2023)
- [Multirate Training of Neural Networks](https://mlsanity.com/paper/2106.10771/) (2021)
- [Quantitative reconstruction of defects in multi-layered bonded composites using fully convolutional network-based ultrasonic inversion](https://mlsanity.com/paper/2109.07284/) (2021)
- [Mystique: Enabling Accurate and Scalable Generation of Production AI Benchmarks](https://mlsanity.com/paper/2301.04122/) (2022)
- [MOFA: A Model Simplification Roadmap for Image Restoration on Mobile Devices](https://mlsanity.com/paper/2308.12494/) (2023)
- [Targeted Adversarial Denoising Autoencoders (TADA) for Neural Time Series Filtration](https://mlsanity.com/paper/2501.04967/) (2025)
- [DNNAbacus: Toward Accurate Computational Cost Prediction for Deep Neural Networks](https://mlsanity.com/paper/2205.12095/) (2022)
- [Efficient Fine-Grained GPU Performance Modeling for Distributed Deep Learning of LLM](https://mlsanity.com/paper/2509.22832/) (2025)
- [Predicting the Performance of a Computing System with Deep Networks](https://mlsanity.com/paper/2302.13638/) (2023)
- [PruMUX: Augmenting Data Multiplexing with Model Compression](https://mlsanity.com/paper/2305.14706/) (2023)
- [Train Faster, Perform Better: Modular Adaptive Training in Over-Parameterized Models](https://mlsanity.com/paper/2405.07527/) (2024)
- [MicroFlow: An Efficient Rust-Based Inference Engine for TinyML](https://mlsanity.com/paper/2409.19432/) (2024)
- [A Neophyte With AutoML: Evaluating the Promises of Automatic Machine Learning Tools](https://mlsanity.com/paper/2101.05840/) (2021)
- [Perception Over Time: Temporal Dynamics for Robust Image Understanding](https://mlsanity.com/paper/2203.06254/) (2022)
- [Towards Optimal VPU Compiler Cost Modeling by using Neural Networks to Infer Hardware Performances](https://mlsanity.com/paper/2205.04586/) (2022)
- [Latency-Constrained DNN Architecture Learning for Edge Systems using Zerorized Batch Normalization](https://mlsanity.com/paper/2607.06922/) (2026)
- [Inference Latency Prediction at the Edge](https://mlsanity.com/paper/2210.02620/) (2022)

## Abstract (arXiv)

Deep learning is rapidly becoming a go-to tool for many artificial intelligence problems due to its ability to outperform other approaches and even humans at many problems. Despite its popularity we are still unable to accurately predict the time it will take to train a deep learning network to solve a given problem. This training time can be seen as the product of the training time per epoch and the number of epochs which need to be performed to reach the desired level of accuracy. Some work has been carried out to predict the training time for an epoch -- most have been based around the assumption that the training time is linearly related to the number of floating point operations required. However, this relationship is not true and becomes exacerbated in cases where other activities start to dominate the execution time. Such as the time to load data from memory or loss of performance due to non-optimal parallel execution. In this work we propose an alternative approach in which we train a deep learning network to predict the execution time for parts of a deep learning network. Timings for these individual parts can then be combined to provide a prediction for the whole execution time. This has advantages over linear approaches as it can model more complex scenarios. But, also, it has the ability to predict execution times for scenarios unseen in the training data. Therefore, our approach can be used not only to infer the execution time for a batch, or entire epoch, but it can also support making a well-informed choice for the appropriate hardware and model.