# From BERT to GPT-3 Codex: Harnessing the Potential of Very Large Language Models for Data Management

arXiv: https://arxiv.org/abs/2306.09339 · HTML: https://mlsanity.com/paper/2306.09339/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Jun 2023.

## Built on by (most cited first, 10 of 10)

- [KeyVideoLLM: Towards Large-scale Video Keyframe Selection](https://mlsanity.com/paper/2407.03104/) (2024)
- [LLM-PBE: Assessing Data Privacy in Large Language Models](https://mlsanity.com/paper/2408.12787/) (2024)
- [Flash-LLM: Enabling Cost-Effective and Highly-Efficient Large Generative Model Inference with Unstructured Sparsity](https://mlsanity.com/paper/2309.10285/) (2023)
- [Synth-Empathy: Towards High-Quality Synthetic Empathy Data](https://mlsanity.com/paper/2407.21669/) (2024)
- [E2ETune: End-to-End Knob Tuning via Fine-tuned Generative Language Model](https://mlsanity.com/paper/2404.11581/) (2024)
- [PAS: Data-Efficient Plug-and-Play Prompt Augmentation System](https://mlsanity.com/paper/2407.06027/) (2024)
- [AGRAG: Advanced Graph-based Retrieval-Augmented Generation for LLMs](https://mlsanity.com/paper/2511.05549/) (2025)
- [Generative Benchmark Creation for Table Union Search](https://mlsanity.com/paper/2308.03883/) (2023)
- [Efficient-Empathy: Towards Efficient and Effective Selection of Empathy Data](https://mlsanity.com/paper/2407.01937/) (2024)
- [Lazy or Efficient? Towards Accessible Eye-Tracking Event Detection Using LLMs](https://mlsanity.com/paper/2604.13243/) (2026)

## Abstract (arXiv)

Large language models have recently advanced the state of the art on many natural language processing benchmarks. The newest generation of models can be applied to a variety of tasks with little to no specialized training. This technology creates various opportunities for applications in the context of data management. The tutorial will introduce participants to basic background on language models, discuss different methods to use language models, and give an overview and short demonstration of available libraries and APIs. Models for generating natural language will be considered as well as models, such as GPT-3 Codex, which complete program code or generate code from natural language instructions. Finally, the tutorial will discuss recent research in the database community that exploits language models in the context of traditional database systems or proposes novel system architectures that are based on them. The tutorial is targeted at database researchers. No prior background on language models is required. The goal of the tutorial is to introduce database researchers to the latest generation of language models, and to their use cases in the domain of data management.