# Low Rank Learning for Offline Query Optimization

arXiv: https://arxiv.org/abs/2504.06399 · HTML: https://mlsanity.com/paper/2504.06399/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 29 of its 91 references that are in the map.
- First posted on arXiv 8 Apr 2025.

## Builds on (most cited first, 29 of 29)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Neural Collaborative Filtering](https://mlsanity.com/paper/1708.05031/) (2017)
- [DeepFM: A Factorization-Machine based Neural Network for CTR Prediction](https://mlsanity.com/paper/1703.04247/) (2017)
- [A Survey of Deep Active Learning](https://mlsanity.com/paper/2009.00236/) (2020)
- [Deep Learning Recommendation Model for Personalization and Recommendation Systems](https://mlsanity.com/paper/1906.00091/) (2019)
- [Deep Learning based Recommender System: A Survey and New Perspectives](https://mlsanity.com/paper/1707.07435/) (2017)
- [Machine Learning for Survival Analysis: A Survey](https://mlsanity.com/paper/1708.04649/) (2017)
- [Matrix Completion and Low-Rank SVD via Fast Alternating Least Squares](https://mlsanity.com/paper/1410.2596/) (2014)
- [Neo: A Learned Query Optimizer](https://mlsanity.com/paper/1904.03711/) (2019)
- [Learned Cardinalities: Estimating Correlated Joins with Deep Learning](https://mlsanity.com/paper/1809.00677/) (2018)
- [Bao: Learning to Steer Query Optimizers](https://mlsanity.com/paper/2004.03814/) (2020)
- [Deep Unsupervised Cardinality Estimation](https://mlsanity.com/paper/1905.04278/) (2019)
- [Learning to Optimize Join Queries With Deep Reinforcement Learning](https://mlsanity.com/paper/1808.03196/) (2018)
- [Lero: A Learning-to-Rank Query Optimizer](https://mlsanity.com/paper/2302.06873/) (2023)
- [Apache Calcite: A Foundational Framework for Optimized Query Processing Over Heterogeneous Data Sources](https://mlsanity.com/paper/1802.10233/) (2018)
- [Deep Reinforcement Learning for Join Order Enumeration](https://mlsanity.com/paper/1803.00055/) (2018)
- [Flow-Loss: Learning Cardinality Estimates That Matter](https://mlsanity.com/paper/2101.04964/) (2021)
- [Balsa: Learning a Query Optimizer Without Expert Demonstrations](https://mlsanity.com/paper/2201.01441/) (2022)
- [Zero-Shot Cost Models for Out-of-the-box Learned Cost Prediction](https://mlsanity.com/paper/2201.00561/) (2022)
- [Learning State Representations for Query Optimization with Deep Reinforcement Learning](https://mlsanity.com/paper/1803.08604/) (2018)
- [ALECE: An Attention-based Learned Cardinality Estimator for SPJ Queries on Dynamic Workloads (Extended)](https://mlsanity.com/paper/2310.05349/) (2023)
- [Building Bridges: Viewing Active Learning from the Multi-Armed Bandit Lens](https://mlsanity.com/paper/1309.6830/) (2013)
- [Matrix completion with queries](https://mlsanity.com/paper/1705.00399/) (2017)
- [Learned Offline Query Planning via Bayesian Optimization](https://mlsanity.com/paper/2502.05256/) (2025)
- [Is Your Learned Query Optimizer Behaving As You Expect? A Machine Learning Perspective](https://mlsanity.com/paper/2309.01551/) (2023)
- [Stage: Query Execution Time Prediction in Amazon Redshift](https://mlsanity.com/paper/2403.02286/) (2024)
- [Guided Automated Learning for query workload re-Optimization](https://mlsanity.com/paper/1901.02049/) (2019)
- [Deploying a Steered Query Optimizer in Production at Microsoft](https://mlsanity.com/paper/2210.13625/) (2022)
- [Grid-AR: A Grid-based Booster for Learned Cardinality Estimation and Range Joins](https://mlsanity.com/paper/2410.07895/) (2024)

## Abstract (arXiv)

Recent deployments of learned query optimizers use expensive neural networks and ad-hoc search policies. To address these issues, we introduce textscLimeQO, a framework for offline query optimization leveraging low-rank learning to efficiently explore alternative query plans with minimal resource usage. By modeling the workload as a partially observed, low-rank matrix, we predict unobserved query plan latencies using purely linear methods, significantly reducing computational overhead compared to neural networks. We formalize offline exploration as an active learning problem, and present simple heuristics that reduces a 3-hour workload to 1.5 hours after just 1.5 hours of exploration. Additionally, we propose a transductive Tree Convolutional Neural Network (TCNN) that, despite higher computational costs, achieves the same workload reduction with only 0.5 hours of exploration. Unlike previous approaches that place expensive neural networks directly in the query processing ``hot'' path, our approach offers a low-overhead solution and a no-regressions guarantee, all without making assumptions about the underlying DBMS. The code is available in hrefhttps://github.com/zixy17/LimeQOhttps://github.com/zixy17/LimeQO.