mlsanityData to 7 Oct 2026

Are Large Language Models Economically Viable for Industry Deployment?

arXiv 2604.19342 ↗First posted 21 Apr 2026cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Generative AI-powered by Large Language Models (LLMs)-is increasingly deployed in industry across healthcare decision support, financial analytics, enterprise retrieval, and conversational automation, where reliability, efficiency, and cost control are critical. In such settings, models must satisfy strict constraints on energy, latency, and hardware utilization-not accuracy alone. Yet prevailing evaluation pipelines remain accuracy-centric, creating a Deployment-Evaluation Gap-the absence of operational and economic criteria in model assessment. To address this gap, we present EDGE-EVAL-a industry-oriented benchmarking framework that evaluates LLMs across their full lifecycle on legacy NVIDIA Tesla T4 GPUs. Benchmarking LLaMA and Qwen variants across three industrial tasks, we introduce five deployment metrics-Economic Break-Even (Nbreak), Intelligence-Per-Watt (IPW ), System Density (rhosys), Cold-Start Tax (Ctax), and Quantization Fidelity (Qret)-capturing profitability, energy efficiency, hardware scaling, serverless feasibility, and compression safety. Our results reveal a clear efficiency frontier-models in the <2B parameter class dominate larger baselines across economic and ecological dimensions. LLaMA-3.2-1B (INT4) achieves ROI break-even in 14 requests (median), delivers 3x higher energy-normalized intelligence than 7B models, and exceeds 6,900 tokens/s/GB under 4-bit quantization. We further uncover an efficiency anomaly-while QLoRA reduces memory footprint, it increases adaptation energy by up to 7x for small models-challenging prevailing assumptions about quantization-aware training in edge deployment.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 15 of its 29 references that are in the map. First posted on arXiv 21 Apr 2026.

Builds on

15 of 29 references.

References · 15

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  2. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  3. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  4. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  5. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  6. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
  7. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,838
  8. 2018Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarizationcited by 1,308
  9. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,234
  10. 2023Enhancing Chat Language Models by Scaling High-quality Instructional Conversationscited by 825
  11. 2022ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformerscited by 517
  12. 2022The Carbon Footprint of Machine Learning Training Will Plateau, Then Shrinkcited by 183
3 more
  1. 2019Dissecting the NVidia Turing T4 GPU via Microbenchmarkingcited by 15
  2. 2025Can Argus Judge Them All? Comparing VLMs Across Domainscited by 2
  3. 2025LLMs on a Budget? Say HOLAcited by 1

Built on by

Not cited yet.