# Fine-Grained Address Segmentation for Attention-Based Variable-Degree Prefetching

arXiv: https://arxiv.org/abs/2205.02269 · HTML: https://mlsanity.com/paper/2205.02269/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 74 references that are in the map.
- First posted on arXiv 1 May 2022.

## Builds on (most cited first, 13 of 13)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [In-Datacenter Performance Analysis of a Tensor Processing Unit](https://mlsanity.com/paper/1704.04760/) (2017)
- [Temporal Convolutional Networks for Action Segmentation and Detection](https://mlsanity.com/paper/1611.05267/) (2016)
- [Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation](https://mlsanity.com/paper/2010.16061/) (2020)
- [AI Benchmark: Running Deep Neural Networks on Android Smartphones](https://mlsanity.com/paper/1810.01109/) (2018)
- [Learning Memory Access Patterns](https://mlsanity.com/paper/1803.02329/) (2018)
- [The GAP Benchmark Suite](https://mlsanity.com/paper/1508.03619/) (2015)
- [Parallel Attention Mechanisms in Neural Machine Translation](https://mlsanity.com/paper/1810.12427/) (2018)
- [NullaNet Tiny: Ultra-low-latency DNN Inference Through Fixed-function Combinational Logic](https://mlsanity.com/paper/2104.05421/) (2021)
- [GraphChallenge.org: Raising the Bar on Graph Analytic Performance](https://mlsanity.com/paper/1805.09675/) (2018)

## Built on by (most cited first, 5 of 5)

- [PaCKD: Pattern-Clustered Knowledge Distillation for Compressing Memory Access Prediction Models](https://mlsanity.com/paper/2402.13441/) (2024)
- [Phases, Modalities, Temporal and Spatial Locality: Domain Specific ML Prefetcher for Accelerating Graph Analytics](https://mlsanity.com/paper/2212.05250/) (2022)
- [Attention, Distillation, and Tabularization: Towards Practical Neural Network-Based Prefetching](https://mlsanity.com/paper/2401.06362/) (2023)
- [TabConv: Low-Computation CNN Inference via Table Lookups](https://mlsanity.com/paper/2404.05872/) (2024)
- [Accurate Open-set Recognition for Memory Workload](https://mlsanity.com/paper/2212.08817/) (2022)

## Abstract (arXiv)

Machine learning algorithms have shown potential to improve prefetching performance by accurately predicting future memory accesses. Existing approaches are based on the modeling of text prediction, considering prefetching as a classification problem for sequence prediction. However, the vast and sparse memory address space leads to large vocabulary, which makes this modeling impractical. The number and order of outputs for multiple cache line prefetching are also fundamentally different from text prediction. We propose TransFetch, a novel way to model prefetching. To reduce vocabulary size, we use fine-grained address segmentation as input. To predict unordered sets of future addresses, we use delta bitmaps for multiple outputs. We apply an attention-based network to learn the mapping between input and output. Prediction experiments demonstrate that address segmentation achieves 26% - 36% higher F1-score than delta inputs and 15% - 24% higher F1-score than page & offset inputs for SPEC 2006, SPEC 2017, and GAP benchmarks. Simulation results show that TransFetch achieves 38.75% IPC improvement compared with no prefetching, outperforming the best-performing rule-based prefetcher BOP by 10.44%, and ML-based prefetcher Voyager by 6.64%.