# Challenges in Deploying Machine Learning: a Survey of Case Studies

arXiv: https://arxiv.org/abs/2011.09926 · HTML: https://mlsanity.com/paper/2011.09926/

## Key facts

- Cited by 152 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Nov 2020.

## Built on by (most cited first, 30 of 152)

- [The Efficiency Misnomer](https://mlsanity.com/paper/2110.12894/) (2021)
- [Exploiting BERT For Multimodal Target Sentiment Classification Through Input Space Translation](https://mlsanity.com/paper/2108.01682/) (2021)
- [Prompt2Model: Generating Deployable Models from Natural Language Instructions](https://mlsanity.com/paper/2308.12261/) (2023)
- [Beyond Preserved Accuracy: Evaluating Loyalty and Robustness of BERT Compression](https://mlsanity.com/paper/2109.03228/) (2021)
- [AutoGluon-Multimodal (AutoMM): Supercharging Multimodal AutoML with Foundation Models](https://mlsanity.com/paper/2404.16233/) (2024)
- [Agent Drift: Quantifying Behavioral Degradation in Multi-Agent LLM Systems Over Extended Interactions](https://mlsanity.com/paper/2601.04170/) (2026)
- [Operationalizing Machine Learning: An Interview Study](https://mlsanity.com/paper/2209.09125/) (2022)
- [A Meta-Summary of Challenges in Building Products with ML Components -- Collecting Experiences from 4758+ Practitioners](https://mlsanity.com/paper/2304.00078/) (2023)
- [Analyzing the Evolution and Maintenance of ML Models on Hugging Face](https://mlsanity.com/paper/2311.13380/) (2023)
- [Knowledge-Adaptation Priors](https://mlsanity.com/paper/2106.08769/) (2021)
- [SPADE: Synthesizing Data Quality Assertions for Large Language Model Pipelines](https://mlsanity.com/paper/2401.03038/) (2024)
- [Table Foundation Models: on knowledge pre-training for tabular learning](https://mlsanity.com/paper/2505.14415/) (2025)
- [On the Limitations of Compute Thresholds as a Governance Strategy](https://mlsanity.com/paper/2407.05694/) (2024)
- [MLOps: A Review](https://mlsanity.com/paper/2308.10908/) (2023)
- [The Pipeline for the Continuous Development of Artificial Intelligence Models -- Current State of Research and Practice](https://mlsanity.com/paper/2301.09001/) (2023)
- [Fedstellar: A Platform for Decentralized Federated Learning](https://mlsanity.com/paper/2306.09750/) (2023)
- [SoK: Pragmatic Assessment of Machine Learning for Network Intrusion Detection](https://mlsanity.com/paper/2305.00550/) (2023)
- [Improving Fairness using Vision-Language Driven Image Augmentation](https://mlsanity.com/paper/2311.01573/) (2023)
- [BERTa'u: Ita'u BERT for digital customer service](https://mlsanity.com/paper/2101.12015/) (2021)
- [Does Data Repair Lead to Fair Models? Curating Contextually Fair Data To Reduce Model Bias](https://mlsanity.com/paper/2110.10389/) (2021)
- [Hierarchical Bayesian Modelling for Knowledge Transfer Across Engineering Fleets via Multitask Learning](https://mlsanity.com/paper/2204.12404/) (2022)
- [Interpretable models for extrapolation in scientific machine learning](https://mlsanity.com/paper/2212.10283/) (2022)
- [Why AI systems don't learn and what to do about it: Lessons on autonomous learning from cognitive science](https://mlsanity.com/paper/2603.15381/) (2026)
- [Looper: An end-to-end ML platform for product decisions](https://mlsanity.com/paper/2110.07554/) (2021)
- [Closing the Loop: A Framework for Trustworthy Machine Learning in Power Systems](https://mlsanity.com/paper/2203.07505/) (2022)
- [Context-Aware Drift Detection](https://mlsanity.com/paper/2203.08644/) (2022)
- [Detecting of a Patient's Condition From Clinical Narratives Using Natural Language Representation](https://mlsanity.com/paper/2104.03969/) (2021)
- [SensiX++: Bringing MLOPs and Multi-tenant Model Serving to Sensory Edge Devices](https://mlsanity.com/paper/2109.03947/) (2021)
- [Quality issues in Machine Learning Software Systems](https://mlsanity.com/paper/2208.08982/) (2022)
- [On Leakage in Machine Learning Pipelines](https://mlsanity.com/paper/2311.04179/) (2023)

## Abstract (arXiv)

In recent years, machine learning has transitioned from a field of academic research interest to a field capable of solving real-world business problems. However, the deployment of machine learning models in production systems can present a number of issues and concerns. This survey reviews published reports of deploying machine learning solutions in a variety of use cases, industries and applications and extracts practical considerations corresponding to stages of the machine learning deployment workflow. By mapping found challenges to the steps of the machine learning deployment workflow we show that practitioners face issues at each stage of the deployment process. The goal of this paper is to lay out a research agenda to explore approaches addressing these challenges.