mlsanityData to 7 Oct 2026

Petals: Collaborative Inference and Fine-tuning of Large Models

arXiv 2209.01188 ↗First posted 2 Sep 2022cs.LG cs.DCcited by 84

Not on the map. Why?

Abstract (arXiv)

Many NLP tasks benefit from using large language models (LLMs) that often have more than 100 billion parameters. With the release of BLOOM-176B and OPT-175B, everyone can download pretrained models of this scale. Still, using these models requires high-end hardware unavailable to many researchers. In some cases, LLMs can be used more affordably via RAM offloading or hosted APIs. However, these techniques have innate limitations: offloading is too slow for interactive inference, while APIs are not flexible enough for research that requires access to weights, attention or logits. In this work, we propose Petals - a system for inference and fine-tuning of large models collaboratively by joining the resources of multiple parties. We demonstrate that this strategy outperforms offloading for very large models, running inference of BLOOM-176B on consumer GPUs with ≈ 1 step per second, which is enough for many interactive LLM applications. Unlike most inference APIs, Petals also natively exposes hidden states of served models, allowing to train and share custom model extensions based on efficient fine-tuning methods.

Key facts. Cited by 84 papers in the map (data through 7 Oct 2026). Builds on 31 of its 52 references that are in the map. First posted on arXiv 2 Sep 2022.

Builds on

31 of 52 references.

References · 31

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  3. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  4. 2020Scaling Laws for Neural Language Modelscited by 6,411
  5. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  6. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  7. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  8. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  9. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,014
  10. 2020GShard: Scaling Giant Models with Conditional Computation and Automatic Shardingcited by 1,601
  11. 2021Improving language models by retrieving from trillions of tokenscited by 1,278
  12. 2021Scaling Language Models: Methods, Analysis & Insights from Training Gophercited by 1,199
19 more
  1. 2021GLaM: Efficient Scaling of Language Models with Mixture-of-Expertscited by 890
  2. 2022Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learningcited by 886
  3. 2022LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scalecited by 881
  4. 2021GPT Understands, Toocited by 715
  5. 2022GPT-NeoX-20B: An Open-Source Autoregressive Language Modelcited by 711
  6. 2021P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and Taskscited by 686
  7. 2021Dynabench: Rethinking Benchmarking in NLPcited by 447
  8. 2020AdapterHub: A Framework for Adapting Transformerscited by 441
  9. 20218-bit Optimizers via Block-wise Quantizationcited by 395
  10. 2020Parameter-Efficient Transfer Learning with Diff Pruningcited by 351
  11. 2021ZeRO-Offload: Democratizing Billion-Scale Model Trainingcited by 339
  12. 2021ZeRO-Infinity: Breaking the GPU Memory Wall for Extreme Scale Deep Learningcited by 253
  13. 2021Training Neural Networks with Fixed Sparse Maskscited by 212
  14. 2021PanGu-α: Large-scale Autoregressive Pretrained Chinese Language Models with Auto-parallel Computationcited by 134
  15. 2021What Changes Can Large-scale Language Models Bring? Intensive Study on HyperCLOVA: Billions-scale Korean Generative Pretrained Transformerscited by 75
  16. 2020Towards Crowdsourced Training of Large Neural Networks using Decentralized Mixture-of-Expertscited by 43
  17. 2022GEMv2: Multilingual NLG Benchmarking in a Single Line of Codecited by 36
  18. 2020Training Large Neural Networks with Constant Memory using a New Execution Algorithmcited by 34
  19. 2022Adapting BigScience Multilingual Model to Unseen Languagescited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 17; 2024: 16; 2025: 22; 2026: 27.

84 papers.

Sort:

By area:

  1. 2023FlexGen: High-Throughput Generative Inference of Large Language Models with a Single GPUcited by 567
  2. 2023Challenges and Applications of Large Language Modelscited by 251
  3. 2023LLM in a flash: Efficient Large Language Model Inference with Limited Memorycited by 170
  4. 2023Fast Distributed Inference Serving for Large Language Modelscited by 143
  5. 2023DiLoCo: Distributed Low-Communication Training of Language Modelscited by 115
  6. 2023Towards Efficient Generative Large Language Model Serving: A Survey from Algorithms to Systemscited by 101
  7. 2022Model Ratatouille: Recycling Diverse Models for Out-of-Distribution Generalizationcited by 97
  8. 2024A Survey of Resource-efficient LLM and Multimodal Foundation Modelscited by 84
  9. 2023Exploring the Benefits of Training Expert Language Models over Instruction Tuningcited by 80
  10. 2023Scaling Expert Language Models with Unsupervised Domain Discoverycited by 59
  11. 2024Understanding LLMs: A Comprehensive Overview from Training to Inferencecited by 58
  12. 2024Helix: Serving Large Language Models over Heterogeneous GPUs and Network via Max-Flowcited by 54
38 more
  1. 2023Knowledge Card: Filling LLMs' Knowledge Gaps with Plug-in Specialized Language Modelscited by 46
  2. 2024NNsight and NDIF: Democratizing Access to Open-Weight Foundation Model Internalscited by 43
  3. 2024How Far Are We From AGI: Are LLMs All We Need?cited by 31
  4. 2024M'elange: Cost Efficient Large Language Model Serving by Exploiting GPU Heterogeneitycited by 30
  5. 2025Streaming DiLoCo with overlapping communication: Towards a Distributed Free Lunchcited by 26
  6. 2024Asynchronous Local-SGD Training for Language Modelingcited by 24
  7. 2025Integrating LLMs with ITS: Recent Advances, Potentials, Challenges, and Future Directionscited by 23
  8. 2025Demystifying Cost-Efficiency in LLM Serving over Heterogeneous GPUscited by 21
  9. 2023LLMeBench: A Flexible Framework for Accelerating LLMs Benchmarkingcited by 20
  10. 2022lo-fi: distributed fine-tuning without communicationcited by 19
  11. 2024LLMCheckup: Conversational Examination of Large Language Models via Interpretability Tools and Self-Explanationscited by 16
  12. 2023Decentralized SGD and Average-direction SAM are Asymptotically Equivalentcited by 15
  13. 2025Seesaw: High-throughput LLM Inference via Model Re-shardingcited by 15
  14. 2025Depth Gives a False Sense of Privacy: LLM Internal States Inversioncited by 14
  15. 2023Quantized Distributed Training of Large Models with Convergence Guaranteescited by 13
  16. 2025Parallax: Efficient LLM Inference Service over Decentralized Environmentcited by 12
  17. 2023Language Models as a Service: Overview of a New Paradigm and its Challengescited by 10
  18. 2025Eager Updates For Overlapped Communication and Computation in DiLoCocited by 10
  19. 2025Distributed LLMs and Multimodal Large Language Models: A Survey on Advances, Challenges, and Future Directionscited by 10
  20. 2024DiPaCo: Distributed Path Compositioncited by 9
  21. 2023Hazards from Increasingly Accessible Fine-Tuning of Downloadable Foundation Modelscited by 7
  22. 2025Understanding Outer Optimizers in Local SGD: Learning Rates, Momentum, and Accelerationcited by 7
  23. 2023A Language Model of Java Methods with Train/Test Deduplicationcited by 6
  24. 2024Activations and Gradients Compression for Model-Parallel Trainingcited by 6
  25. 2024No Need to Talk: Asynchronous Mixture of Language Modelscited by 5
  26. 2025Privacy-Preserving LLM Embedding Transmission for End-Cloud Collaborationcited by 5
  27. 2024Efficient Sparse Attention needs Adaptive Token Releasecited by 4
  28. 2024Achieving Peak Performance for Large Language Models: A Systematic Reviewcited by 4
  29. 2025DeServe: Towards Affordable Offline LLM Inference via Decentralizationcited by 4
  30. 2025A Cost-Effective Near-Storage Processing Solution for Offline Inference of Long-Context LLMscited by 4
  31. 2023Who's the Best Detective? LLMs vs. MLs in Detecting Incoherent Fourth Grade Math Answerscited by 3
  32. 2023LinguaLinked: A Distributed Large Language Model Inference System for Mobile Devicescited by 3
  33. 2024eFedLLM: Efficient LLM Inference Based on Federated Learningcited by 3
  34. 2026FlexSpec: Frozen Drafts Meet Evolving Targets in Edge-Cloud Collaborative LLM Speculative Decodingcited by 3
  35. 2026A Survey on the Green Development of Large Models: From Resource-Efficient Architectures to Hardware-Software Co-Designcited by 3
  36. 2025A Satellite-Ground Synergistic Large Vision-Language Model System for Earth Observationcited by 2
  37. 2025DICE: Data Influence Cascade in Decentralized Learningcited by 2
  38. 2025Adaptive and Resource-efficient Agentic AI Systems for Mobile and Embedded Devices: A Surveycited by 2

50 most cited of 84 All 84 (JSON)