# Reducing Load Latency with Cache Level Prediction

arXiv: https://arxiv.org/abs/2103.14808 · HTML: https://mlsanity.com/paper/2103.14808/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Mar 2021.

## Built on by (most cited first, 4 of 4)

- [Hermes: Accelerating Long-Latency Load Requests via Perceptron-Based Off-Chip Load Prediction](https://mlsanity.com/paper/2209.00188/) (2022)
- [Athena: Synergizing Data Prefetching and Off-Chip Prediction via Online Reinforcement Learning](https://mlsanity.com/paper/2601.17615/) (2026)
- [A Two Level Neural Approach Combining Off-Chip Prediction with Adaptive Prefetch Filtering](https://mlsanity.com/paper/2403.15181/) (2024)
- [Machine Learning-Driven Intelligent Memory System Design: From On-Chip Caches to Storage](https://mlsanity.com/paper/2603.14583/) (2026)

## Abstract (arXiv)

High load latency that results from deep cache hierarchies and relatively slow main memory is an important limiter of single-thread performance. Data prefetch helps reduce this latency by fetching data up the hierarchy before it is requested by load instructions. However, data prefetching has shown to be imperfect in many situations. We propose cache-level prediction to complement prefetchers. Our method predicts which memory hierarchy level a load will access allowing the memory loads to start earlier, and thereby saves many cycles. The predictor provides high prediction accuracy at the cost of just one cycle added latency to L1 misses. Experimental results show speedup of 7.8% on generic, graph, and HPC applications over a baseline with aggressive prefetchers.