TranSQL+: Serving Large Language Models with SQL on Low-Resource Hardware
Not on the map. Why?
Abstract (arXiv)
Deploying Large Language Models (LLMs) on resource-constrained devices remains challenging due to limited memory, lack of GPUs, and the complexity of existing runtimes. In this paper, we introduce TranSQL+, a template-based code generator that translates LLM computation graphs into pure SQL queries for execution in relational databases. Without relying on external libraries, TranSQL+, leverages mature database features, such as vectorized execution and out-of-core processing, for efficient inference. We further propose a row-to-column (ROW2COL) optimization that improves join efficiency in matrix operations. Evaluated on Llama3-8B and DeepSeekMoE models, TranSQL+ achieves up to 20x lower prefill latency and 4x higher decoding speed compared to DeepSpeed Inference and Llama.cpp in low-memory and CPU-only configurations. Our results highlight relational databases as a practical environment for LLMs on low-resource hardware.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 21 of its 52 references that are in the map. First posted on arXiv 5 Feb 2025.
Builds on
21 of 52 references.
References · 21
- 2017Attention Is All You Needcited by 59,932
- 2024The Llama 3 Herd of Modelscited by 14,192
- 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
- 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
- 2024Qwen2 Technical Reportcited by 2,076
- 2020GLU Variants Improve Transformercited by 1,724
- 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,311
- 2024DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Modelscited by 834
- 2019Fast Transformer Decoding: One Write-Head is All You Needcited by 700
- 2024DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Servingcited by 595
- 2023FlexGen: High-Throughput Generative Inference of Large Language Models with a Single GPUcited by 567
- 2024InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agentscited by 476
9 more
- 2023LMSYS-Chat-1M: A Large-Scale Real-World LLM Conversation Datasetcited by 449
- 2013Learning Factored Representations in a Deep Mixture of Expertscited by 275
- 2023LLM in a flash: Efficient Large Language Model Inference with Limited Memorycited by 170
- 2024PowerInfer-2: Fast Large Language Model Inference on a Smartphonecited by 101
- 2024Med-MoE: Mixture of Domain-Specific Experts for Lightweight Medical Vision-Language Modelscited by 43
- 2024BLADE: Enhancing Black-box Large Language Models with Small Domain-Specific Modelscited by 15
- 2016Towards Linear Algebra over Normalized Datacited by 12
- 2019Declarative Recursive Computation on an RDBMS, or, Why You Should Use a Database For Distributed Machine Learningcited by 12
- 2023The Duck's Brain: Training and Inference of Neural Networks in Modern Database Enginescited by 2
Built on by
Citing papers per year: 2026: 3.
3 papers.