# Learning State Representations for Query Optimization with Deep Reinforcement Learning

arXiv: https://arxiv.org/abs/1803.08604 · HTML: https://mlsanity.com/paper/1803.08604/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Mar 2018.

## Built on by (most cited first, 14 of 14)

- [DB-BERT: a Database Tuning Tool that "Reads the Manual"](https://mlsanity.com/paper/2112.10925/) (2021)
- [A Unified Transferable Model for ML-Enhanced DBMS](https://mlsanity.com/paper/2105.02418/) (2021)
- [CARMI: A Cache-Aware Learned Index with a Cost-based Construction Algorithm](https://mlsanity.com/paper/2103.00858/) (2021)
- [Efficient Deep Learning Pipelines for Accurate Cost Estimations Over Large Scale Query Workload](https://mlsanity.com/paper/2103.12465/) (2021)
- [InTune: Reinforcement Learning-based Data Pipeline Optimization for Deep Recommendation Models](https://mlsanity.com/paper/2308.08500/) (2023)
- [TERC: A Transfer Entropy Redundancy Criterion for State Variable Selection in Reinforcement Learning](https://mlsanity.com/paper/2401.11512/) (2024)
- [A Pluggable Learned Index Method via Sampling and Gap Insertion](https://mlsanity.com/paper/2101.00808/) (2021)
- [Phoebe: A Learning-based Checkpoint Optimizer](https://mlsanity.com/paper/2110.02313/) (2021)
- [JoinGym: An Efficient Query Optimization Environment for Reinforcement Learning](https://mlsanity.com/paper/2307.11704/) (2023)
- [Multi-agent Databases via Independent Learning](https://mlsanity.com/paper/2205.14323/) (2022)
- [BitE : Accelerating Learned Query Optimization in a Mixed-Workload Environment](https://mlsanity.com/paper/2306.00845/) (2023)
- [Low Rank Learning for Offline Query Optimization](https://mlsanity.com/paper/2504.06399/) (2025)
- [OntoTune: Ontology-Driven Learning for Query Optimization with Convolutional Models](https://mlsanity.com/paper/2511.06780/) (2025)
- [RELOAD: A Robust and Efficient Learned Query Optimizer for Database Systems](https://mlsanity.com/paper/2604.14725/) (2026)

## Abstract (arXiv)

Deep reinforcement learning is quickly changing the field of artificial intelligence. These models are able to capture a high level understanding of their environment, enabling them to learn difficult dynamic tasks in a variety of domains. In the database field, query optimization remains a difficult problem. Our goal in this work is to explore the capabilities of deep reinforcement learning in the context of query optimization. At each state, we build queries incrementally and encode properties of subqueries through a learned representation. The challenge here lies in the formation of the state transition function, which defines how the current subquery state combines with the next query operation (action) to yield the next state. As a first step in this direction, we focus the state representation problem and the formation of the state transition function. We describe our approach and show preliminary results. We further discuss how we can use the state representation to improve query optimization using reinforcement learning.