# Sibyl: Adaptive and Extensible Data Placement in Hybrid Storage Systems Using Online Reinforcement Learning

arXiv: https://arxiv.org/abs/2205.07394 · HTML: https://mlsanity.com/paper/2205.07394/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 188 references that are in the map.
- First posted on arXiv 15 May 2022.

## Builds on (most cited first, 16 of 16)

- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [A Distributional Perspective on Reinforcement Learning](https://mlsanity.com/paper/1707.06887/) (2017)
- [Programmatically Interpretable Reinforcement Learning](https://mlsanity.com/paper/1804.02477/) (2018)
- [Explainable Reinforcement Learning Through a Causal Lens](https://mlsanity.com/paper/1905.10958/) (2019)
- [Explainable Reinforcement Learning: A Survey](https://mlsanity.com/paper/2005.06247/) (2020)
- [Hyperparameter Selection for Offline Reinforcement Learning](https://mlsanity.com/paper/2007.09055/) (2020)
- [Chip Placement with Deep Reinforcement Learning](https://mlsanity.com/paper/2004.10746/) (2020)
- [Why Deep Neural Networks for Function Approximation?](https://mlsanity.com/paper/1610.04161/) (2016)
- [Toward Interpretable Deep Reinforcement Learning with Linear Model U-Trees](https://mlsanity.com/paper/1807.05887/) (2018)
- [Interestingness Elements for Explainable Reinforcement Learning: Understanding Agents' Capabilities and Limitations](https://mlsanity.com/paper/1912.09007/) (2019)
- [An Imitation Learning Approach for Cache Replacement](https://mlsanity.com/paper/2006.16239/) (2020)
- [Pythia: A Customizable Hardware Prefetching Framework Using Online Reinforcement Learning](https://mlsanity.com/paper/2109.12021/) (2021)
- [Error Characterization, Mitigation, and Recovery in Flash Memory Based Solid-State Drives](https://mlsanity.com/paper/1706.08642/) (2017)
- [Data-driven battery operation for energy arbitrage using rainbow deep reinforcement learning](https://mlsanity.com/paper/2106.06061/) (2021)
- [Intelligent Architectures for Intelligent Computing Systems](https://mlsanity.com/paper/2012.12381/) (2020)
- [Reducing Solid-State Drive Read Latency by Optimizing Read-Retry](https://mlsanity.com/paper/2104.09611/) (2021)

## Built on by (most cited first, 8 of 8)

- [An Experimental Evaluation of Machine Learning Training on a Real Processing-in-Memory System](https://mlsanity.com/paper/2207.07886/) (2022)
- [SwiftRL: Towards Efficient Reinforcement Learning on Real Processing-In-Memory Systems](https://mlsanity.com/paper/2405.03967/) (2024)
- [Hierarchical Resource Partitioning on Modern GPUs: A Reinforcement Learning Approach](https://mlsanity.com/paper/2405.08754/) (2024)
- [Network Contention-Aware Cluster Scheduling with Reinforcement Learning](https://mlsanity.com/paper/2310.20209/) (2023)
- [A Bring-Your-Own-Model Approach for ML-Driven Storage Placement in Warehouse-Scale Computers](https://mlsanity.com/paper/2501.05651/) (2025)
- [Athena: Synergizing Data Prefetching and Off-Chip Prediction via Online Reinforcement Learning](https://mlsanity.com/paper/2601.17615/) (2026)
- [Is Intelligence the Right Direction in New OS Scheduling for Multiple Resources in Cloud Environments?](https://mlsanity.com/paper/2504.15021/) (2025)
- [Machine Learning-Driven Intelligent Memory System Design: From On-Chip Caches to Storage](https://mlsanity.com/paper/2603.14583/) (2026)

## Abstract (arXiv)

Hybrid storage systems (HSS) use multiple different storage devices to provide high and scalable storage capacity at high performance. Recent research proposes various techniques that aim to accurately identify performance-critical data to place it in a "best-fit" storage device. Unfortunately, most of these techniques are rigid, which (1) limits their adaptivity to perform well for a wide range of workloads and storage device configurations, and (2) makes it difficult for designers to extend these techniques to different storage system configurations (e.g., with a different number or different types of storage devices) than the configuration they are designed for. We introduce Sibyl, the first technique that uses reinforcement learning for data placement in hybrid storage systems. Sibyl observes different features of the running workload as well as the storage devices to make system-aware data placement decisions. For every decision it makes, Sibyl receives a reward from the system that it uses to evaluate the long-term performance impact of its decision and continuously optimizes its data placement policy online. We implement Sibyl on real systems with various HSS configurations. Our results show that Sibyl provides 21.6%/19.9% performance improvement in a performance-oriented/cost-oriented HSS configuration compared to the best previous data placement technique. Our evaluation using an HSS configuration with three different storage devices shows that Sibyl outperforms the state-of-the-art data placement policy by 23.9%-48.2%, while significantly reducing the system architect's burden in designing a data placement mechanism that can simultaneously incorporate three storage devices. We show that Sibyl achieves 80% of the performance of an oracle policy that has complete knowledge of future access patterns while incurring a very modest storage overhead of only 124.4 KiB.