On the Sustainability of AI Inferences in the Edge
Not on the map. Why?
Abstract (arXiv)
The proliferation of the Internet of Things (IoT) and its cutting-edge AI-enabled applications (e.g., autonomous vehicles and smart industries) combine two paradigms: data-driven systems and their deployment on the edge. Usually, edge devices perform inferences to support latency-critical applications. In addition to the performance of these resource-constrained edge devices, their energy usage is a critical factor in adopting and deploying edge applications. Examples of such devices include Raspberry Pi (RPi), Intel Neural Compute Stick (INCS), NVIDIA Jetson nano (NJn), and Google Coral USB (GCU). Despite their adoption in edge deployment for AI inferences, there is no study on their performance and energy usage for informed decision-making on the device and model selection to meet the demands of applications. This study fills the gap by rigorously characterizing the performance of traditional, neural networks, and large language models on the above-edge devices. Specifically, we analyze trade-offs among model F1 score, inference time, inference power, and memory usage. Hardware and framework optimization, along with external parameter tuning of AI models, can balance between model performance and resource usage to realize practical edge AI deployments.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 12 of its 40 references that are in the map. First posted on arXiv 30 Jul 2025.
Builds on
12 of 40 references.
References · 12
- 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,899
- 2019TinyBERT: Distilling BERT for Natural Language Understandingcited by 1,167
- 2023OpenAssistant Conversations -- Democratizing Large Language Model Alignmentcited by 721
- 2024Large Language Models: A Surveycited by 525
- 2022The Carbon Footprint of Machine Learning Training Will Plateau, Then Shrinkcited by 184
- 2024LLM-Inference-Bench: Inference Benchmarking of Large Language Models on AI Acceleratorscited by 32
- 2023Green Edge AI: A Contemporary Surveycited by 25
- 2024EdgeLLM: A Highly Efficient CPU-FPGA Heterogeneous Edge Accelerator for Large Language Modelscited by 21
- 2019pCAMP: Performance Comparison of Machine Learning Packages on the Edgescited by 12
- 2025Understanding the Performance and Power of LLM Inferencing on Edge Acceleratorscited by 7
- 2023Edge Devices Inference Performance Comparisoncited by 3
- 2024DynaSplit: A Hardware-Software Co-Design Framework for Energy-Aware Inference on Edgecited by 2
Built on by
Citing papers per year: 2026: 1.
1 paper.