Triple-Bottom-Line Sustainability of Language Models for Edge AI: A Comparison Between SLMs and Quantized LLMs
Not on the map. Why?
Abstract (arXiv)
Edge-AI model selection is commonly driven by one isolated metric - accuracy, latency, memory, energy, or safety, even though a deployable language model must balance all five. Our work focuses on answering the question whether na- tively trained small language models (SLMs) or large language models (LLMs) compressed through post-training quantization offer the more sustainable edge- deployment trade-off. We introduce a reproducible Holistic Sustainability Score (HSS) organized around the triple bottom line: an economic pillar for capability and systems efficiency, an environmental pillar for operational GPU energy and a social pillar for harmful-prompt robustness. Five BF16 SLMs and five LLMs under different quantization approaches - BF16, INT8, NF4 4-bit, GPTQ 4-bit, and GGUF Q4 produce 30 measured configurations. Capability is assessed on five zero-shot benchmarks; efficiency uses latency, throughput, peak VRAM and energy; and safety is approximated by attack success rate on five harmful prompts. Qwen3-30B-A3B/GGUF Q4 ranks first in the combined pool (93.38), followed by Mistral-Small-24B/GGUF Q4 (92.40), while Phi-4-mini/BF16 is the highest- ranked SLM in that pool (89.49). Thus, the hypothesis that native SLMs must be the most sustainable edge choice is not supported universally; optimized quantized LLMs can win overall, while SLMs remain competitive through lower resource demand. Quantization is a systems-level choice rather than a monotonic precision- efficiency trade-off and HSS remains relative to its comparison pool and proxy definitions.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 15 of its 18 references that are in the map. First posted on arXiv 1 Sep 2026.
Builds on
15 of 18 references.
References · 15
- 2021Training Verifiers to Solve Math Word Problemscited by 10,109
- 2025Qwen3 Technical Reportcited by 8,820
- 2020Measuring Massive Multitask Language Understandingcited by 7,963
- 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
- 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,100
- 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
- 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,975
- 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,919
- 2025Gemma 3 Technical Reportcited by 1,824
- 2019Energy and Policy Considerations for Deep Learning in NLPcited by 1,254
- 2022LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scalecited by 906
- 2019Green AIcited by 627
3 more
- 2024JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Modelscited by 536
- 2025Sustainable LLM Inference for Edge AI: Evaluating Quantized LLMs for Energy Efficiency, Output Accuracy, and Inference Latencycited by 34
- 2024Exploring the Trade-Offs: Quantization Methods, Task Difficulty, and Model Size in Large Language Models From Edge to Giantcited by 14
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Sometimes Painful but Certainly Promising: Feasibility and Trade-offs of Language Model Inference at the Edge
- 2025Optimizing LLMs for Resource-Constrained Environments: A Survey of Model Compression Techniques
- 2025Addressing the sustainable AI trilemma: a case study on LLM agents and RAG
- 2025Cognitive Edge Computing: A Comprehensive Survey on Optimizing Large Models and AI Agents for Pervasive Deployment
- 2026Are Large Language Models Economically Viable for Industry Deployment?
- 2025CarbonCall: Sustainability-Aware Function Calling for Large Language Models on Edge Devices
- 2025A Systematic Evaluation of On-Device LLMs: Quantization, Performance, and Resources
- 2025Efficient Edge LLMs Deployment via HessianAware Quantization and CPU GPU Collaborative