# Serving Deep Learning Model in Relational Databases

arXiv: https://arxiv.org/abs/2310.04696 · HTML: https://mlsanity.com/paper/2310.04696/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 84 references that are in the map.
- First posted on arXiv 7 Oct 2023.

## Builds on (most cited first, 28 of 28)

- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [Billion-scale similarity search with GPUs](https://mlsanity.com/paper/1702.08734/) (2017)
- [REALM: Retrieval-Augmented Language Model Pre-Training](https://mlsanity.com/paper/2002.08909/) (2020)
- [A Tutorial on Bayesian Optimization](https://mlsanity.com/paper/1807.02811/) (2018)
- [Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimization](https://mlsanity.com/paper/1603.06560/) (2016)
- [FlexGen: High-Throughput Generative Inference of Large Language Models with a Single GPU](https://mlsanity.com/paper/2303.06865/) (2023)
- [Do Deep Generative Models Know What They Don't Know?](https://mlsanity.com/paper/1810.09136/) (2018)
- [DeepSpeed Inference: Enabling Efficient Inference of Transformer Models at Unprecedented Scale](https://mlsanity.com/paper/2207.00032/) (2022)
- [Deep Learning Recommendation Model for Personalization and Recommendation Systems](https://mlsanity.com/paper/1906.00091/) (2019)
- [Clipper: A Low-Latency Online Prediction Serving System](https://mlsanity.com/paper/1612.03079/) (2016)
- [Neural Approaches to Conversational AI](https://mlsanity.com/paper/1809.08267/) (2018)
- [DocPrompting: Generating Code by Retrieving the Docs](https://mlsanity.com/paper/2207.05987/) (2022)
- [Tensor Comprehensions: Framework-Agnostic High-Performance Machine Learning Abstractions](https://mlsanity.com/paper/1802.04730/) (2018)
- [Learned Cardinalities: Estimating Correlated Joins with Deep Learning](https://mlsanity.com/paper/1809.00677/) (2018)
- [LSH Ensemble: Internet-Scale Domain Search](https://mlsanity.com/paper/1603.07410/) (2016)
- [PRETZEL: Opening the Black Box of Machine Learning Prediction Serving Systems](https://mlsanity.com/paper/1810.06115/) (2018)
- [Extending Relational Query Processing with ML Inference](https://mlsanity.com/paper/1911.00231/) (2019)
- [Serving Deep Learning Models with Deduplication from Relational Databases](https://mlsanity.com/paper/2201.10442/) (2022)
- [Declarative Recursive Computation on an RDBMS, or, Why You Should Use a Database For Distributed Machine Learning](https://mlsanity.com/paper/1904.11121/) (2019)
- [Tensor Relational Algebra for Machine Learning System Design](https://mlsanity.com/paper/2009.00524/) (2020)
- [Optimizing Tensor Programs on Flexible Storage](https://mlsanity.com/paper/2210.06267/) (2022)
- [DISTAL: The Distributed Tensor Algebra Compiler](https://mlsanity.com/paper/2203.08069/) (2022)
- [PlinyCompute: A Platform for High-Performance, Distributed, Data-Intensive Tool Development](https://mlsanity.com/paper/1711.05573/) (2017)
- [Bring Your Own Codegen to Deep Learning Compiler](https://mlsanity.com/paper/2105.03215/) (2021)
- [Pangea: Monolithic Distributed Storage for Data Analytics](https://mlsanity.com/paper/1808.06094/) (2018)
- [A Comparison of Decision Forest Inference Platforms from A Database Perspective](https://mlsanity.com/paper/2302.04430/) (2023)
- [Accelerating Deep Learning Classification with Error-controlled Approximate-key Caching](https://mlsanity.com/paper/2112.06671/) (2021)
- [Benchmark of DNN Model Search at Deployment Time](https://mlsanity.com/paper/2206.00188/) (2022)

## Built on by (most cited first, 2 of 2)

- [Access Paths for Efficient Ordering with Large Language Models](https://mlsanity.com/paper/2509.00303/) (2025)
- [InferF: Declarative Factorization of AI/ML Inferences over Joins](https://mlsanity.com/paper/2511.20489/) (2025)

## Abstract (arXiv)

Serving deep learning (DL) models on relational data has become a critical requirement across diverse commercial and scientific domains, sparking growing interest recently. In this visionary paper, we embark on a comprehensive exploration of representative architectures to address the requirement. We highlight three pivotal paradigms: The state-of-the-art DL-centric architecture offloads DL computations to dedicated DL frameworks. The potential UDF-centric architecture encapsulates one or more tensor computations into User Defined Functions (UDFs) within the relational database management system (RDBMS). The potential relation-centric architecture aims to represent a large-scale tensor computation through relational operators. While each of these architectures demonstrates promise in specific use scenarios, we identify urgent requirements for seamless integration of these architectures and the middle ground in-between these architectures. We delve into the gaps that impede the integration and explore innovative strategies to close them. We present a pathway to establish a novel RDBMS for enabling a broad class of data-intensive DL inference applications.