mlsanityData to 7 Oct 2026

Desiderata for next generation of ML model serving

arXiv 2210.14665 ↗First posted 26 Oct 2022cs.LG cs.SEcited by 5

Not on the map. Why?

Abstract (arXiv)

Inference is a significant part of ML software infrastructure. Despite the variety of inference frameworks available, the field as a whole can be considered in its early days. This position paper puts forth a range of important qualities that next generation of inference platforms should be aiming for. We present our rationale for the importance of each quality, and discuss ways to achieve it in practice. We propose to focus on data-centricity as the overarching design pattern which enables smarter ML system deployment and operation at scale.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 15 of its 46 references that are in the map. First posted on arXiv 26 Oct 2022.

Builds on

15 of 46 references.

References · 15

  1. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,250
  2. 2016Stealing Machine Learning Models via Prediction APIscited by 602
  3. 2021Carbon Emissions and Large Neural Network Trainingcited by 561
  4. 2021A White Paper on Neural Network Quantizationcited by 453
  5. 2022DeepSpeed Inference: Enabling Efficient Inference of Transformer Models at Unprecedented Scalecited by 403
  6. 2021Sustainable AI: Environmental Implications, Challenges and Opportunitiescited by 242
  7. 2020Challenges in Deploying Machine Learning: a Survey of Case Studiescited by 151
  8. 2022I Know What You Trained Last Summer: A Survey on Stealing Machine Learning Models and Defencescited by 75
  9. 2018Algorithms that Remember: Model Inversion Attacks and Data Protection Lawcited by 52
  10. 2020Monitoring and explainability of models in productioncited by 23
  11. 2022Context-Aware Drift Detectioncited by 9
  12. 2022DeeprETA: An ETA Post-processing System at Scalecited by 7
3 more
  1. 2022An Empirical Evaluation of Flow Based Programming in the Machine Learning Deployment Contextcited by 5
  2. 2014Beyond Batch Processing: Towards Real-Time and Streaming Big Datacited by 2
  3. 2022Assessing Software Privacy using the Privacy Flow-Graphcited by 2

Built on by

Cited by year

Citing papers per year: 2023: 5.

5 papers.

  1. 2023Dataflow graphs as complete causal graphscited by 5
  2. 2023Reconciling High Accuracy, Cost-Efficiency, and Low Latency of Inference Serving Systemscited by 5
  3. 2023Machine Learning Systems: A Survey from a Data-Oriented Perspectivecited by 3
  4. 2023IPA: Inference Pipeline Adaptation to Achieve High Accuracy and Cost-Efficiencycited by 2
  5. 2023Causal fault localisation in dataflow systemscited by 1