mlsanityData to 7 Oct 2026

RIBBON: Cost-Effective and QoS-Aware Deep Learning Model Inference using a Diverse Pool of Cloud Computing Instances

arXiv 2207.11434 ↗First posted 23 Jul 2022cs.DC cs.LGcited by 1

Not on the map. Why?

Abstract (arXiv)

Deep learning model inference is a key service in many businesses and scientific discovery processes. This paper introduces RIBBON, a novel deep learning inference serving system that meets two competing objectives: quality-of-service (QoS) target and cost-effectiveness. The key idea behind RIBBON is to intelligently employ a diverse set of cloud computing instances (heterogeneous instances) to meet the QoS target and maximize cost savings. RIBBON devises a Bayesian Optimization-driven strategy that helps users build the optimal set of heterogeneous instances for their model inference service needs on cloud computing platforms -- and, RIBBON demonstrates its superiority over existing approaches of inference serving systems using homogeneous instance pools. RIBBON saves up to 16% of the inference service cost for different learning models including emerging deep learning recommender system models and drug-discovery enabling models.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 28 of its 97 references that are in the map. First posted on arXiv 23 Jul 2022.

Builds on

28 of 97 references.

References · 28

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  3. 2018YOLOv3: An Incremental Improvementcited by 2,652
  4. 2016Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learningcited by 1,509
  5. 2017Neural Collaborative Filteringcited by 1,108
  6. 2016Wide & Deep Learning for Recommender Systemscited by 745
  7. 2017In-Datacenter Performance Analysis of a Tensor Processing Unitcited by 610
  8. 2017Deep Interest Network for Click-Through Rate Predictioncited by 541
  9. 2017Deep learning for universal linear embeddings of nonlinear dynamicscited by 491
  10. 2018Deep Interest Evolution Network for Click-Through Rate Predictioncited by 336
  11. 2019MLPerf Inference Benchmarkcited by 195
  12. 2016Clipper: A Low-Latency Online Prediction Serving Systemcited by 191
16 more
  1. 2019Edge AI: On-Demand Accelerating Deep Neural Network Inference via Edge Computingcited by 150
  2. 2019Deep Learning vs. Traditional Computer Visioncited by 94
  3. 2019The Architectural Implications of Facebook's DNN-based Personalized Recommendationcited by 83
  4. 2020Deep Learning COVID-19 Features on CXR using Limited Training Data Setscited by 68
  5. 2017Dealing with Integer-valued Variables in Bayesian Optimization with Gaussian Processescited by 65
  6. 2020DeepRecSys: A System for Optimizing End-To-End At-scale Neural Recommendation Inferencecited by 54
  7. 2020Deep Learning Training in Facebook Data Centers: Design of Scale-up and Scale-out Systemscited by 37
  8. 2018Deep Learning Inference in Facebook Data Centers: Characterization, Performance Optimizations and Hardware Implicationscited by 35
  9. 2018DLHub: Model and Data Serving for Sciencecited by 17
  10. 2020Optimizing Deep Learning Recommender Systems' Training On CPU Cluster Architecturescited by 17
  11. 2019Multi-objective Bayesian optimisation with preferences over objectivescited by 16
  12. 2020Cross-Stack Workload Characterization of Deep Recommendation Systemscited by 11
  13. 2017Low-Level Augmented Bayesian Optimization for Finding the Best Cloud VMcited by 9
  14. 2019Bayesian Optimization for Multi-objective Optimization and Multi-point Searchcited by 8
  15. 2018Micky: A Cheaper Alternative for Selecting Cloud Instancescited by 6
  16. 2018Performance Analysis and Modeling of Video Transcoding Using Heterogeneous Cloud Servicescited by 2

Built on by

Cited by year

Citing papers per year: 2022: 1.

1 paper.

  1. 2022KAIROS: Building Cost-Efficient Machine Learning Inference Systems with Heterogeneous Cloud Resourcescited by 3