# The Tensor Data Platform: Towards an AI-centric Database System

arXiv: https://arxiv.org/abs/2211.02753 · HTML: https://mlsanity.com/paper/2211.02753/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 42 references that are in the map.
- First posted on arXiv 4 Nov 2022.

## Builds on (most cited first, 13 of 13)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Neural-Symbolic VQA: Disentangling Reasoning from Vision and Language Understanding](https://mlsanity.com/paper/1810.02338/) (2018)
- [Differentially Private Query Release Through Adaptive Projection](https://mlsanity.com/paper/2103.06641/) (2021)
- [On Learning from Label Proportions](https://mlsanity.com/paper/1402.5902/) (2014)
- [Weakly Supervised Classification in High Energy Physics](https://mlsanity.com/paper/1702.00414/) (2017)
- [Learning with Algorithmic Supervision via Continuous Relaxations](https://mlsanity.com/paper/2110.05651/) (2021)
- [A Tensor Compiler for Unified Machine Learning Prediction Serving](https://mlsanity.com/paper/2010.04804/) (2020)
- [End-to-end Optimization of Machine Learning Prediction Queries](https://mlsanity.com/paper/2206.00136/) (2022)
- [Extending Relational Query Processing with ML Inference](https://mlsanity.com/paper/1911.00231/) (2019)
- [Declarative Recursive Computation on an RDBMS, or, Why You Should Use a Database For Distributed Machine Learning](https://mlsanity.com/paper/1904.11121/) (2019)
- [Query Processing on Tensor Computation Runtimes](https://mlsanity.com/paper/2203.01877/) (2022)
- [Share the Tensor Tea: How Databases can Leverage the Machine Learning Ecosystem](https://mlsanity.com/paper/2209.04579/) (2022)
- [Weakly supervised clustering: Learning fine-grained signals from coarse labels](https://mlsanity.com/paper/1310.1363/) (2013)

## Built on by (most cited first, 3 of 3)

- [Optimizing Context-Enhanced Relational Joins](https://mlsanity.com/paper/2312.01476/) (2023)
- [Bullion: A Column Store for Machine Learning](https://mlsanity.com/paper/2404.08901/) (2024)
- [Towards Effective Orchestration of AI x DB Workloads](https://mlsanity.com/paper/2603.03772/) (2026)

## Abstract (arXiv)

Database engines have historically absorbed many of the innovations in data processing, adding features to process graph data, XML, object oriented, and text among many others. In this paper, we make the case that it is time to do the same for AI -- but with a twist! While existing approaches have tried to achieve this by integrating databases with external ML tools, in this paper we claim that achieving a truly AI-centric database requires moving the DBMS engine, at its core, from a relational to a tensor abstraction. This allows us to: (1) support multi-modal data processing such as images, videos, audio, text as well as relational; (2) leverage the wellspring of innovation in HW and runtimes for tensor computation; and (3) exploit automatic differentiation to enable a novel class of "trainable" queries that can learn to perform a task. To support the above scenarios, we introduce TDP: a system that builds upon our prior work mapping relational queries to tensors. Thanks to a tighter integration with the tensor runtime, TDP is able to provide a broader coverage of new emerging scenarios requiring access to multi-modal data and automatic differentiation.