# LLMs as On-demand Customizable Service

arXiv: https://arxiv.org/abs/2401.16577 · HTML: https://mlsanity.com/paper/2401.16577/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 15 references that are in the map.
- First posted on arXiv 29 Jan 2024.

## Builds on (most cited first, 8 of 8)

- [Knowledge Distillation: A Survey](https://mlsanity.com/paper/2006.05525/) (2020)
- [Learn to Grow: A Continual Structure Learning Framework for Overcoming Catastrophic Forgetting](https://mlsanity.com/paper/1904.00310/) (2019)
- [Data Poisoning Attacks Against Federated Learning Systems](https://mlsanity.com/paper/2007.08432/) (2020)
- [Continual Pre-training of Language Models](https://mlsanity.com/paper/2302.03241/) (2023)
- [LAMOL: LAnguage MOdeling for Lifelong Language Learning](https://mlsanity.com/paper/1909.03329/) (2019)
- [AutoML to Date and Beyond: Challenges and Opportunities](https://mlsanity.com/paper/2010.10777/) (2020)
- [Free-riders in Federated Learning: Attacks and Defenses](https://mlsanity.com/paper/1911.12560/) (2019)
- [Federated Learning of N-gram Language Models](https://mlsanity.com/paper/1910.03432/) (2019)

## Built on by (most cited first, 1 of 1)

- [Accelerating Cross-Model LLM Inference via Attention Map Reuse](https://mlsanity.com/paper/2610.14688/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) have demonstrated remarkable language understanding and generation capabilities. However, training, deploying, and accessing these models pose notable challenges, including resource-intensive demands, extended training durations, and scalability issues. To address these issues, we introduce a concept of hierarchical, distributed LLM architecture that aims at enhancing the accessibility and deployability of LLMs across heterogeneous computing platforms, including general-purpose computers (e.g., laptops) and IoT-style devices (e.g., embedded systems). By introducing a "layered" approach, the proposed architecture enables on-demand accessibility to LLMs as a customizable service. This approach also ensures optimal trade-offs between the available computational resources and the user's application needs. We envision that the concept of hierarchical LLM will empower extensive, crowd-sourced user bases to harness the capabilities of LLMs, thereby fostering advancements in AI technology in general.