Computing in the Era of Large Generative Models: From Cloud-Native to AI-Native
Not on the map. Why?
Abstract (arXiv)
In this paper, we investigate the intersection of large generative AI models and cloud-native computing architectures. Recent large models such as ChatGPT, while revolutionary in their capabilities, face challenges like escalating costs and demand for high-end GPUs. Drawing analogies between large-model-as-a-service (LMaaS) and cloud database-as-a-service (DBaaS), we describe an AI-native computing paradigm that harnesses the power of both cloud-native technologies (e.g., multi-tenancy and serverless computing) and advanced machine learning runtime (e.g., batched LoRA inference). These joint efforts aim to optimize costs-of-goods-sold (COGS) and improve resource accessibility. The journey of merging these two domains is just at the beginning and we hope to stimulate future research and development in this area.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 28 of its 76 references that are in the map. First posted on arXiv 17 Jan 2024.
Builds on
28 of 76 references.
References · 28
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
- 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
- 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
- 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
- 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
- 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
- 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,014
- 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,551
- 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Frameworkcited by 1,761
- 2022Fast Inference from Transformers via Speculative Decodingcited by 1,615
- 2019Machine Unlearningcited by 1,020
16 more
- 2023Accelerating Large Language Model Decoding with Speculative Samplingcited by 975
- 2016Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphscited by 853
- 2021Efficient Large-Scale Language Model Training on GPU Clusters Using Megatron-LMcited by 621
- 2023FlexGen: High-Throughput Generative Inference of Large Language Models with a Single GPUcited by 567
- 2021Carbon Emissions and Large Neural Network Trainingcited by 561
- 2017Deep Learning Scaling is Predictable, Empiricallycited by 556
- 2022DeepSpeed-MoE: Advancing Mixture-of-Experts Inference and Training to Power Next-Generation AI Scalecited by 400
- 2023Multi-Agent Collaboration: Harnessing the Power of Intelligent LLM Agentscited by 329
- 2020Serving DNNs like Clockwork: Performance Predictability from the Bottom Upcited by 119
- 2022Decentralized Training of Foundation Models in Heterogeneous Environmentscited by 87
- 2017TensorFlow-Serving: Flexible, High-Performance ML Servingcited by 85
- 2023Punica: Multi-Tenant LoRA Servingcited by 80
- 2020Heterogeneity-Aware Cluster Scheduling Policies for Deep Learning Workloadscited by 56
- 2023High-Throughput Vector Similarity Search in Knowledge Graphscited by 19
- 2017Occupy the Cloud: Distributed Computing for the 99%cited by 17
- 2022Serving and Optimizing Machine Learning Workflows on Heterogeneous Infrastructurescited by 12
Built on by
Citing papers per year: 2025: 1; 2026: 1.
2 papers.