mlsanityData to 14 Oct 2026

Hera: A Heterogeneity-Aware Multi-Tenant Inference Server for Personalized Recommendations

arXiv 2302.11750 ↗First posted 23 Feb 2023cs.DC cs.IR cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

While providing low latency is a fundamental requirement in deploying recommendation services, achieving high resource utility is also crucial in cost-effectively maintaining the datacenter. Co-locating multiple workers of a model is an effective way to maximize query-level parallelism and server throughput, but the interference caused by concurrent workers at shared resources can prevent server queries from meeting its SLA. Hera utilizes the heterogeneous memory requirement of multi-tenant recommendation models to intelligently determine a productive set of co-located models and its resource allocation, providing fast response time while achieving high throughput. We show that Hera achieves an average 37.3% improvement in effective machine utilization, enabling 26% reduction in required servers, significantly improving upon the baseline recommedation inference server.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 22 of its 41 references that are in the map. First posted on arXiv 23 Feb 2023.

Builds on

22 of 41 references.

References · 22

  1. 2017Neural Collaborative Filteringcited by 1,111
  2. 2016Wide & Deep Learning for Recommender Systemscited by 747
  3. 2017Deep Interest Network for Click-Through Rate Predictioncited by 544
  4. 2018Deep Interest Evolution Network for Click-Through Rate Predictioncited by 338
  5. 2019Deep Learning Recommendation Model for Personalization and Recommendation Systemscited by 327
  6. 2017TensorFlow-Serving: Flexible, High-Performance ML Servingcited by 85
  7. 2019The Architectural Implications of Facebook's DNN-based Personalized Recommendationcited by 83
  8. 2020Distributed Hierarchical GPU Parameter Server for Massive Scale Deep Learning Ads Systemscited by 60
  9. 2020DeepRecSys: A System for Optimizing End-To-End At-scale Neural Recommendation Inferencecited by 54
  10. 2019RecNMP: Accelerating Personalized Recommendation with Near-Memory Processingcited by 49
  11. 2019TensorDIMM: A Practical Near-Memory Processing Architecture for Embeddings and Tensor Operations in Deep Learningcited by 43
  12. 2018Deep Learning Inference in Facebook Data Centers: Characterization, Performance Optimizations and Hardware Implicationscited by 36
10 more
  1. 2021RecSSD: Near Data Processing for Solid State Drive Based Recommendation Inferencecited by 35
  2. 2019PREMA: A Predictive Multi-task Scheduling Algorithm For Preemptible Neural Processing Unitscited by 30
  3. 2020Understanding Capacity-Driven Scale-Out Neural Recommendation Inferencecited by 29
  4. 2020Centaur: A Chiplet-based, Hybrid Sparse-Dense Accelerator for Personalized Recommendationscited by 21
  5. 2019Beyond Human-Level Accuracy: Computational Challenges in Deep Learningcited by 19
  6. 2020Optimizing Deep Learning Recommender Systems' Training On CPU Cluster Architecturescited by 17
  7. 2020Tensor Casting: Co-Designing Algorithm-Architecture for Personalized Recommendation Trainingcited by 13
  8. 2021JIZHI: A Fast and Cost-Effective Model-As-A-Service System for Web-Scale Online Inference at Baiducited by 10
  9. 2021RecPipe: Co-designing Models and Hardware to Jointly Optimize Recommendation Quality and Performancecited by 8
  10. 2021Low-Precision Hardware Architectures Meet Recommendation Model Inference at Scalecited by 7

Built on by

Not cited yet.