# Pythia: A Customizable Hardware Prefetching Framework Using Online Reinforcement Learning

arXiv: https://arxiv.org/abs/2109.12021 · HTML: https://mlsanity.com/paper/2109.12021/

## Key facts

- Cited by 17 papers in the map (data through 7 Oct 2026).
- Builds on 8 of its 145 references that are in the map.
- First posted on arXiv 24 Sep 2021.

## Builds on (most cited first, 8 of 8)

- [ConfuciuX: Autonomous Hardware Resource Assignment for DNN Accelerators using Reinforcement Learning](https://mlsanity.com/paper/2009.02010/) (2020)
- [Learning Memory Access Patterns](https://mlsanity.com/paper/1803.02329/) (2018)
- [An Imitation Learning Approach for Cache Replacement](https://mlsanity.com/paper/2006.16239/) (2020)
- [Learning Execution through Neural Code Fusion](https://mlsanity.com/paper/1906.07181/) (2019)
- [Accelerating Deep Learning Inference via Learned Caches](https://mlsanity.com/paper/2101.07344/) (2021)
- [Improving Branch Prediction By Modeling Global History with Convolutional Neural Networks](https://mlsanity.com/paper/1906.09889/) (2019)
- [DSPatch: Dual Spatial Pattern Prefetcher](https://mlsanity.com/paper/1910.03075/) (2019)
- [Branch Prediction as a Reinforcement Learning Problem: Why, How and Case Studies](https://mlsanity.com/paper/2106.13429/) (2021)

## Built on by (most cited first, 17 of 17)

- [Sibyl: Adaptive and Extensible Data Placement in Hybrid Storage Systems Using Online Reinforcement Learning](https://mlsanity.com/paper/2205.07394/) (2022)
- [Hermes: Accelerating Long-Latency Load Requests via Perceptron-Based Off-Chip Load Prediction](https://mlsanity.com/paper/2209.00188/) (2022)
- [Puppeteer: A Random Forest-based Manager for Hardware Prefetchers across the Memory Hierarchy](https://mlsanity.com/paper/2201.12027/) (2022)
- [DeepSketch: A New Machine Learning-Based Reference Search Technique for Post-Deduplication Delta Compression](https://mlsanity.com/paper/2202.10584/) (2022)
- [SwiftRL: Towards Efficient Reinforcement Learning on Real Processing-In-Memory Systems](https://mlsanity.com/paper/2405.03967/) (2024)
- [Harmonia: Enhancing Data Placement and Migration in Hybrid Storage Systems via Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2503.20507/) (2025)
- [Coordinated Reinforcement Learning Prefetching Architecture for Multicore Systems](https://mlsanity.com/paper/2509.10719/) (2025)
- [A Joint Learning Approach to Hardware Caching and Prefetching](https://mlsanity.com/paper/2510.10862/) (2025)
- [Athena: Synergizing Data Prefetching and Off-Chip Prediction via Online Reinforcement Learning](https://mlsanity.com/paper/2601.17615/) (2026)
- [Agentic Architect: An Agentic AI Framework for Architecture Design Exploration and Optimization](https://mlsanity.com/paper/2604.25083/) (2026)
- [Cello: Efficient Computer Systems Optimization with Predictive Early Termination and Censored Regression](https://mlsanity.com/paper/2204.04831/) (2022)
- [DeePref: Deep Reinforcement Learning For Video Prefetching In Content Delivery Networks](https://mlsanity.com/paper/2310.07881/) (2023)
- [Artificial Intelligence-Driven Network-on-Chip Design Space Exploration: Neural Network Architectures for Design](https://mlsanity.com/paper/2512.07877/) (2025)
- [Rudder: Steering Prefetching in Distributed GNN Training using LLM Agents](https://mlsanity.com/paper/2602.23556/) (2026)
- [Machine Learning-Driven Intelligent Memory System Design: From On-Chip Caches to Storage](https://mlsanity.com/paper/2603.14583/) (2026)
- [ReLMXEL: Adaptive RL-Based Memory Controller with Explainable Energy and Latency Optimization](https://mlsanity.com/paper/2603.17309/) (2026)
- [ArchAgent v2: A Case Study with the Data Prefetching Championship](https://mlsanity.com/paper/2608.09874/) (2026)

## Abstract (arXiv)

Past research has proposed numerous hardware prefetching techniques, most of which rely on exploiting one specific type of program context information (e.g., program counter, cacheline address) to predict future memory accesses. These techniques either completely neglect a prefetcher's undesirable effects (e.g., memory bandwidth usage) on the overall system, or incorporate system-level feedback as an afterthought to a system-unaware prefetch algorithm. We show that prior prefetchers often lose their performance benefit over a wide range of workloads and system configurations due to their inherent inability to take multiple different types of program context and system-level feedback information into account while prefetching. In this paper, we make a case for designing a holistic prefetch algorithm that learns to prefetch using multiple different types of program context and system-level feedback information inherent to its design. To this end, we propose Pythia, which formulates the prefetcher as a reinforcement learning agent. For every demand request, Pythia observes multiple different types of program context information to make a prefetch decision. For every prefetch decision, Pythia receives a numerical reward that evaluates prefetch quality under the current memory bandwidth usage. Pythia uses this reward to reinforce the correlation between program context information and prefetch decision to generate highly accurate, timely, and system-aware prefetch requests in the future. Our extensive evaluations using simulation and hardware synthesis show that Pythia outperforms multiple state-of-the-art prefetchers over a wide range of workloads and system configurations, while incurring only 1.03% area overhead over a desktop-class processor and no software changes in workloads. The source code of Pythia can be freely downloaded from https://github.com/CMU-SAFARI/Pythia.