# GenAI at the Edge: Comprehensive Survey on Empowering Edge Devices

arXiv: https://arxiv.org/abs/2502.15816 · HTML: https://mlsanity.com/paper/2502.15816/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Feb 2025.

## Built on by (most cited first, 14 of 14)

- [KVCOMM: Online Cross-context KV-cache Communication for Efficient LLM-based Multi-agent Systems](https://mlsanity.com/paper/2510.12872/) (2025)
- [Embodied Foundation Models at the Edge: A Survey of Deployment Constraints and Mitigation Strategies](https://mlsanity.com/paper/2603.16952/) (2026)
- [MedMambaLite: Hardware-Aware Mamba for Medical Image Classification](https://mlsanity.com/paper/2508.05049/) (2025)
- [ShadowNPU: System and Algorithm Co-design for NPU-Centric On-Device LLM Inference](https://mlsanity.com/paper/2508.16703/) (2025)
- [ECVL-ROUTER: Scenario-Aware Routing for Vision-Language Models](https://mlsanity.com/paper/2510.27256/) (2025)
- [Generative Artificial Intelligence for Internet of Things Computing: A Systematic Survey](https://mlsanity.com/paper/2504.07635/) (2025)
- [EdgeWisePersona: A Dataset for On-Device User Profiling from Natural Language Interactions](https://mlsanity.com/paper/2505.11417/) (2025)
- [EdgeNavMamba: Mamba Optimized Object Detection for Energy Efficient Edge Devices](https://mlsanity.com/paper/2510.14946/) (2025)
- [EVA: Accelerating LLM Decoding via an Efficient Vector Quantization Architecture](https://mlsanity.com/paper/2605.24144/) (2026)
- [Bridging the Perception Gap: A Lightweight Coarse-to-Fine Architecture for Edge Audio Systems](https://mlsanity.com/paper/2601.15676/) (2026)
- [CBPNet: A Continual Backpropagation Prompt Network for Alleviating Plasticity Loss on Edge Devices](https://mlsanity.com/paper/2509.15785/) (2025)
- [A Unified Model and Document Representation for On-Device Retrieval-Augmented Generation](https://mlsanity.com/paper/2604.14403/) (2026)
- [Large Models for Small Devices: Recent Advances and Empirical Analysis of Edge AI Deployment](https://mlsanity.com/paper/2608.15693/) (2026)
- [Vortex: Bridging Extreme Compression and Efficient LLM Inference](https://mlsanity.com/paper/2609.12208/) (2026)

## Abstract (arXiv)

Generative Artificial Intelligence (GenAI) applies models and algorithms such as Large Language Model (LLM) and Foundation Model (FM) to generate new data. GenAI, as a promising approach, enables advanced capabilities in various applications, including text generation and image processing. In current practice, GenAI algorithms run mainly on the cloud server, leading to high latency and raising security concerns. Consequently, these challenges encourage the deployment of GenAI algorithms directly on edge devices. However, the large size of such models and their significant computational resource requirements pose obstacles when deploying them in resource-constrained systems. This survey provides a comprehensive overview of recent proposed techniques that optimize GenAI for efficient deployment on resource-constrained edge devices. For this aim, this work highlights three main categories for bringing GenAI to the edge: software optimization, hardware optimization, and frameworks. The main takeaways for readers of this survey will be a clear roadmap to design, implement, and refine GenAI systems for real-world implementation on edge devices.