# Joint Foundation Model Caching and Inference of Generative AI Services for Edge Intelligence

arXiv: https://arxiv.org/abs/2305.12130 · HTML: https://mlsanity.com/paper/2305.12130/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 May 2023.

## Built on by (most cited first, 9 of 9)

- [Mobile Edge Intelligence for Large Language Models: A Contemporary Survey](https://mlsanity.com/paper/2407.18921/) (2024)
- [Filling the Missing: Exploring Generative AI for Enhanced Federated Learning over Heterogeneous Mobile Edge Devices](https://mlsanity.com/paper/2310.13981/) (2023)
- [Satellite Edge Artificial Intelligence with Large Models: Architectures and Technologies](https://mlsanity.com/paper/2504.01676/) (2025)
- [Privacy-Aware Joint DNN Model Deployment and Partitioning Optimization for Collaborative Edge Inference Services](https://mlsanity.com/paper/2502.16091/) (2025)
- [A Survey of Blockchain, Artificial Intelligence, and Edge Computing for Web 3.0](https://mlsanity.com/paper/2311.13731/) (2023)
- [Resource-Efficient Generative AI Model Deployment in Mobile Edge Networks](https://mlsanity.com/paper/2409.05303/) (2024)
- [Distillation-Enabled Knowledge Alignment for Generative Semantic Communications of AIGC Images](https://mlsanity.com/paper/2506.19893/) (2025)
- [A Model Aware AIGC Task Offloading Algorithm in IIoT Edge Computing](https://mlsanity.com/paper/2507.11560/) (2025)
- [Multi-Agentic AI for Fairness-Aware and Accelerated Multi-modal Large Model Inference in Real-world Mobile Edge Networks](https://mlsanity.com/paper/2602.07215/) (2026)

## Abstract (arXiv)

With the rapid development of artificial general intelligence (AGI), various multimedia services based on pretrained foundation models (PFMs) need to be effectively deployed. With edge servers that have cloud-level computing power, edge intelligence can extend the capabilities of AGI to mobile edge networks. However, compared with cloud data centers, resource-limited edge servers can only cache and execute a small number of PFMs, which typically consist of billions of parameters and require intensive computing power and GPU memory during inference. To address this challenge, in this paper, we propose a joint foundation model caching and inference framework that aims to balance the tradeoff among inference latency, accuracy, and resource consumption by managing cached PFMs and user requests efficiently during the provisioning of generative AI services. Specifically, considering the in-context learning ability of PFMs, a new metric named the Age of Context (AoC), is proposed to model the freshness and relevance between examples in past demonstrations and current service requests. Based on the AoC, we propose a least context caching algorithm to manage cached PFMs at edge servers with historical prompts and inference results. The numerical results demonstrate that the proposed algorithm can reduce system costs compared with existing baselines by effectively utilizing contextual information.