# Learning Semantics, Not Addresses: Runtime Neural Prefetching for Far Memory

arXiv: https://arxiv.org/abs/2506.00384 · HTML: https://mlsanity.com/paper/2506.00384/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 35 references that are in the map.
- First posted on arXiv 31 May 2025.

## Builds on (most cited first, 14 of 14)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://mlsanity.com/paper/2312.00752/) (2023)
- [RoFormer: Enhanced Transformer with Rotary Position Embedding](https://mlsanity.com/paper/2104.09864/) (2021)
- [Retentive Network: A Successor to Transformer for Large Language Models](https://mlsanity.com/paper/2307.08621/) (2023)
- [Graph Retrieval-Augmented Generation: A Survey](https://mlsanity.com/paper/2408.08921/) (2024)
- [Learning Memory Access Patterns](https://mlsanity.com/paper/1803.02329/) (2018)
- [The GAP Benchmark Suite](https://mlsanity.com/paper/1508.03619/) (2015)
- [A neural network memory prefetcher using semantic locality](https://mlsanity.com/paper/1804.00478/) (2018)
- [BOLT: A Practical Binary Optimizer for Data Centers and Beyond](https://mlsanity.com/paper/1807.06735/) (2018)
- [Effectively Prefetching Remote Memory with Leap](https://mlsanity.com/paper/1911.09829/) (2019)
- [DEX: Scalable Range Indexing on Disaggregated Memory [Extended Version]](https://mlsanity.com/paper/2405.14502/) (2024)
- [Attention, Distillation, and Tabularization: Towards Practical Neural Network-Based Prefetching](https://mlsanity.com/paper/2401.06362/) (2023)

## Abstract (arXiv)

Memory prefetching has long boosted CPU caches and is increasingly vital for far-memory systems, where large portions of memory are offloaded to cheaper, remote tiers. While effective prefetching requires accurate prediction of future accesses, prior ML approaches have been limited to simulation or small-scale hardware. We introduce FarSight, the first Linux-based far-memory system to leverage deep learning by decoupling application semantics from runtime memory layout. This separation enables offline-trained models to predict access patterns over a compact ordinal vocabulary, which are resolved at runtime through lightweight mappings. Across four data-intensive workloads, FarSight delivers up to 3.6x higher performance than the state-of-the-art.