mlsanityData to 7 Oct 2026

Challenges in Deploying Machine Learning: a Survey of Case Studies

arXiv 2011.09926 ↗First posted 18 Nov 2020cs.LGcited by 151

Not on the map. Why?

Abstract (arXiv)

In recent years, machine learning has transitioned from a field of academic research interest to a field capable of solving real-world business problems. However, the deployment of machine learning models in production systems can present a number of issues and concerns. This survey reviews published reports of deploying machine learning solutions in a variety of use cases, industries and applications and extracts practical considerations corresponding to stages of the machine learning deployment workflow. By mapping found challenges to the steps of the machine learning deployment workflow we show that practitioners face issues at each stage of the deployment process. The goal of this paper is to lay out a research agenda to explore approaches addressing these challenges.

Key facts. Cited by 151 papers in the map (data through 7 Oct 2026). First posted on arXiv 18 Nov 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 22; 2022: 12; 2023: 37; 2024: 30; 2025: 24; 2026: 26.

151 papers.

Sort:

By area:

  1. 2021The Efficiency Misnomercited by 98
  2. 2021Exploiting BERT For Multimodal Target Sentiment Classification Through Input Space Translationcited by 41
  3. 2023Prompt2Model: Generating Deployable Models from Natural Language Instructionscited by 40
  4. 2021Beyond Preserved Accuracy: Evaluating Loyalty and Robustness of BERT Compressioncited by 38
  5. 2024AutoGluon-Multimodal (AutoMM): Supercharging Multimodal AutoML with Foundation Modelscited by 29
  6. 2026Agent Drift: Quantifying Behavioral Degradation in Multi-Agent LLM Systems Over Extended Interactionscited by 26
  7. 2022Operationalizing Machine Learning: An Interview Studycited by 25
  8. 2023A Meta-Summary of Challenges in Building Products with ML Components -- Collecting Experiences from 4758+ Practitionerscited by 22
  9. 2023Analyzing the Evolution and Maintenance of ML Models on Hugging Facecited by 22
  10. 2021Knowledge-Adaptation Priorscited by 20
  11. 2024SPADE: Synthesizing Data Quality Assertions for Large Language Model Pipelinescited by 19
  12. 2024On the Limitations of Compute Thresholds as a Governance Strategycited by 17
38 more
  1. 2023MLOps: A Reviewcited by 16
  2. 2025Table Foundation Models: on knowledge pre-training for tabular learningcited by 15
  3. 2023The Pipeline for the Continuous Development of Artificial Intelligence Models -- Current State of Research and Practicecited by 14
  4. 2023Fedstellar: A Platform for Decentralized Federated Learningcited by 14
  5. 2023SoK: Pragmatic Assessment of Machine Learning for Network Intrusion Detectioncited by 13
  6. 2023Improving Fairness using Vision-Language Driven Image Augmentationcited by 13
  7. 2021BERTa'u: Ita'u BERT for digital customer servicecited by 11
  8. 2021Does Data Repair Lead to Fair Models? Curating Contextually Fair Data To Reduce Model Biascited by 11
  9. 2022Hierarchical Bayesian Modelling for Knowledge Transfer Across Engineering Fleets via Multitask Learningcited by 11
  10. 2022Interpretable models for extrapolation in scientific machine learningcited by 11
  11. 2021Looper: An end-to-end ML platform for product decisionscited by 9
  12. 2022Closing the Loop: A Framework for Trustworthy Machine Learning in Power Systemscited by 9
  13. 2022Context-Aware Drift Detectioncited by 9
  14. 2021Detecting of a Patient's Condition From Clinical Narratives Using Natural Language Representationcited by 8
  15. 2026Why AI systems don't learn and what to do about it: Lessons on autonomous learning from cognitive sciencecited by 8
  16. 2021SensiX++: Bringing MLOPs and Multi-tenant Model Serving to Sensory Edge Devicescited by 7
  17. 2022Quality issues in Machine Learning Software Systemscited by 7
  18. 2023On Leakage in Machine Learning Pipelinescited by 7
  19. 2024Shaving Weights with Occam's Razor: Bayesian Sparsification for Neural Networks Using the Marginal Likelihoodcited by 7
  20. 2024Understanding the Fundamental Design Decisions of Retrieval-Augmented Generation Systemscited by 7
  21. 2021ML-EXray: Visibility into ML Deployment on the Edgecited by 6
  22. 2024Semi-Supervised Learning for Deep Causal Generative Modelscited by 6
  23. 2024Continuous fake media detection: adapting deepfake detectors to new generative techniquescited by 6
  24. 2024Automatic Data Labeling for Software Vulnerability Prediction Models: How Far Are We?cited by 6
  25. 2024Deepfake Media Forensics: State of the Art and Challenges Aheadcited by 6
  26. 2021Using AntiPatterns to avoid MLOps Mistakescited by 5
  27. 2021Machine Learning for Fraud Detection in E-Commerce: A Research Agendacited by 5
  28. 2021Test for non-negligible adverse shiftscited by 5
  29. 2022Reinforcement Learning in Practice: Opportunities and Challengescited by 5
  30. 2022An Empirical Evaluation of Flow Based Programming in the Machine Learning Deployment Contextcited by 5
  31. 2022Desiderata for next generation of ML model servingcited by 5
  32. 2023AI for Science: An Emerging Agendacited by 5
  33. 2023Dataflow graphs as complete causal graphscited by 5
  34. 2023Diffusion Denoised Smoothing for Certified and Adversarial Robust Out-Of-Distribution Detectioncited by 5
  35. 2023Generative User-Experience Research for Developing Domain-specific Natural Language Processing Applicationscited by 5
  36. 2023Modyn: Data-Centric Machine Learning Pipeline Orchestrationcited by 5
  37. 2023The State of Documentation Practices of Third-party Machine Learning Models and Datasetscited by 5
  38. 2024Human-in-the-Loop Synthetic Text Data Inspection with Provenance Trackingcited by 5

50 most cited of 151 All 151 (JSON)