mlsanityData to 14 Oct 2026

Performance Trade-offs of Optimizing Small Language Models for E-Commerce

arXiv 2510.21970 ↗First posted 24 Oct 2025cs.AI cs.CLcited by 1

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) offer state-of-the-art performance in natural language understanding and generation tasks. However, the deployment of leading commercial models for specialized tasks, such as e-commerce, is often hindered by high computational costs, latency, and operational expenses. This paper investigates the viability of smaller, open-weight models as a resource-efficient alternative. We present a methodology for optimizing a one-billion-parameter Llama 3.2 model for multilingual e-commerce intent recognition. The model was fine-tuned using Quantized Low-Rank Adaptation (QLoRA) on a synthetically generated dataset designed to mimic real-world user queries. Subsequently, we applied post-training quantization techniques, creating GPU-optimized (GPTQ) and CPU-optimized (GGUF) versions. Our results demonstrate that the specialized 1B model achieves 99% accuracy, matching the performance of the significantly larger GPT-4.1 model. A detailed performance analysis revealed critical, hardware-dependent trade-offs: while 4-bit GPTQ reduced VRAM usage by 41%, it paradoxically slowed inference by 82% on an older GPU architecture (NVIDIA T4) due to dequantization overhead. Conversely, GGUF formats on a CPU achieved a speedup of up to 18x in inference throughput and a reduction of over 90% in RAM consumption compared to the FP16 baseline. We conclude that small, properly optimized open-weight models are not just a viable but a more suitable alternative for domain-specific applications, offering state-of-the-art accuracy at a fraction of the computational cost.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 38 of its 45 references that are in the map. First posted on arXiv 24 Oct 2025.

Builds on

38 of 45 references.

References · 38

  1. 2017Attention Is All You Needcited by 60,361
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2025Qwen3 Technical Reportcited by 8,820
  4. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
  5. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,724
  6. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,919
  7. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,386
  8. 2024DynamoLLM: Designing LLM Inference Clusters for Performance and Energy Efficiencycited by 112
  9. 2024AgentInstruct: Toward Generative Teaching with Agentic Flowscited by 89
  10. 2023Goat: Fine-tuned LLaMA Outperforms GPT-4 on Arithmetic Taskscited by 84
  11. 2025How Hungry is AI? Benchmarking Energy, Water, and Carbon Footprint of LLM Inferencecited by 76
  12. 2025Synthetic Data Generation Using Large Language Models: Advances in Text and Codecited by 61
26 more
  1. 2025Energy Considerations of Large Language Model Inference and Efficiency Optimizationscited by 55
  2. 2023Intriguing Properties of Quantization at Scalecited by 49
  3. 2024LoRA Land: 310 Fine-tuned LLMs that Rival GPT-4, A Technical Reportcited by 47
  4. 2024DoRA: Enhancing Parameter-Efficient Fine-Tuning with Dynamic Rank Distributioncited by 43
  5. 2025Quantization Meets Reasoning: Exploring LLM Low-Bit Quantization Degradation for Mathematical Reasoningcited by 37
  6. 2024Evaluating Language Models as Synthetic Data Generatorscited by 33
  7. 2025ShoppingBench: A Real-World Intent-Grounded Shopping Benchmark for LLM-based Agentscited by 28
  8. 2025StructEval: Benchmarking LLMs' Capabilities to Generate Structural Outputscited by 25
  9. 2024The Price of Prompting: Profiling Energy Use in Large Language Models Inferencecited by 23
  10. 2024StructuredRAG: JSON Response Formatting with Large Language Modelscited by 22
  11. 2025Characterizing LLM Inference Energy-Performance Tradeoffs across Workloads and GPU Scalingcited by 19
  12. 2025Think Inside the JSON: Reinforcement Strategy for Strict LLM Schema Adherencecited by 16
  13. 2025CLONE: Customizing LLMs for Efficient Latency-Aware Inference at the Edgecited by 16
  14. 2024Exploring the Trade-Offs: Quantization Methods, Task Difficulty, and Model Size in Large Language Models From Edge to Giantcited by 14
  15. 2025Benchmarking Post-Training Quantization in LLMs: Comprehensive Taxonomy, Unified Evaluation, and Comparative Analysiscited by 12
  16. 2024LiLiuM: eBay's Large Language Models for e-commercecited by 10
  17. 2025MetaSynth: Meta-Prompting-Driven Agentic Scaffolds for Diverse Synthetic Data Generationcited by 10
  18. 2025TeLLMe: An Energy-Efficient Ternary LLM Accelerator for Prefilling and Decoding on Edge FPGAscited by 10
  19. 2025Systematic Characterization of LLM Quantization: A Performance, Energy, and Quality Perspectivecited by 9
  20. 2025Quantized but Deceptive? A Multi-Dimensional Truthfulness Evaluation of Quantized LLMscited by 9
  21. 2024A Usage-centric Take on Intent Understanding in E-Commercecited by 6
  22. 2025Fine-Tune an SLM or Prompt an LLM? The Case of Generating Low-Code Workflowscited by 6
  23. 2025Beyond Scale: Small Language Models are Comparable to GPT-4 in Mental Health Understandingcited by 6
  24. 2024LLM-Based Robust Product Classification in Commerce and Compliancecited by 5
  25. 2024BioMistral-NLU: Towards More Generalizable Medical Language Understanding through Instruction Tuningcited by 4
  26. 2025Narrowing the Gap: Supervised Fine-Tuning of Open-Source LLMs as a Viable Alternative to Proprietary Models for Pedagogical Toolscited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Unifying Data, Memory, and Compute Efficiency in LLM training: A Surveycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Compass-v3: Scaling Domain-Specific LLMs for Multilingual E-Commerce in Southeast Asia
  2. 2024A Comprehensive Study on Quantization Techniques for Large Language Models
  3. 2024A Comprehensive Evaluation of Quantization Strategies for Large Language Models
  4. 2025Survey of Specialized Large Language Model
  5. 2024Evaluating the Generalization Ability of Quantized LLMs: Benchmark, Analysis, and Toolbox
  6. 2023Dissecting the Runtime Performance of the Training, Fine-tuning, and Inference of Large Language Models
  7. 2023Understanding the Impact of Post-Training Quantization on Large Language Models
  8. 2024EC-Guide: A Comprehensive E-Commerce Guide for Instruction Tuning and Quantization