mlsanityData to 14 Oct 2026

AI Application Operations -- A Socio-Technical Framework for Data-driven Organizations

arXiv 2601.06061 ↗First posted 23 Dec 2025cs.CY cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

We outline a comprehensive framework for artificial intelligence (AI) Application Operations (AIAppOps), based on real-world experiences from diverse organizations. Data-driven projects pose additional challenges to organizations due to their dependency on data across the development and operations cycles. To aid organizations in dealing with these challenges, we present a framework outlining the main steps and roles involved in going from idea to production for data-driven solutions. The data dependency of these projects entails additional requirements on continuous monitoring and feedback, as deviations can emerge in any process step. Therefore, the framework embeds monitoring not merely as a safeguard, but as a unifying feedback mechanism that drives continuous improvement, compliance, and sustained value realization-anchored in both statistical and formal assurance methods that extend runtime verification concepts from safety-critical AI to organizational operations. The proposed framework is structured across core technical processes and supporting services to guide both new initiatives and maturing AI programs.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 16 of its 43 references that are in the map. First posted on arXiv 23 Dec 2025.

Builds on

16 of 43 references.

References · 16

  1. 2017On Calibration of Modern Neural Networkscited by 4,960
  2. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,926
  3. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,385
  4. 2016Concrete Problems in AI Safetycited by 1,642
  5. 2017Towards A Rigorous Science of Interpretable Machine Learningcited by 1,416
  6. 2022Machine Learning Operations (MLOps): Overview, Definition, and Architecturecited by 128
  7. 2016On the Safety of Machine Learning: Cyber-Physical Systems, Decision Sciences, and Data Productscited by 52
  8. 2015Robust Online Monitoring of Signal Temporal Logiccited by 39
  9. 2022Data-Centric Artificial Intelligencecited by 31
  10. 2020Data Readiness Reportcited by 17
  11. 2023Robust Conformal Prediction for STL Runtime Verification under Distribution Shiftcited by 15
  12. 2017Data Readiness Levelscited by 13
4 more
  1. 2024"We Have No Idea How Models will Behave in Production until Production": How Engineers Operationalize Machine Learningcited by 5
  2. 2025Monitoring Machine Learning Systems: A Multivocal Literature Reviewcited by 4
  3. 2018Verified Runtime Validation for Partially Observable Hybrid Systemscited by 3
  4. 2024Exploratory Visual Analysis for Increasing Data Readiness in Artificial Intelligence Projectscited by 1

Built on by

Not cited yet.