# RTop-K: Ultra-Fast Row-Wise Top-K Selection for Neural Network Acceleration on GPUs

arXiv: https://arxiv.org/abs/2409.00822 · HTML: https://mlsanity.com/paper/2409.00822/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Sep 2024.

## Built on by (most cited first, 8 of 8)

- [SonicMoE: Accelerating MoE with IO and Tile-aware Optimizations](https://mlsanity.com/paper/2512.14080/) (2025)
- [MamNet: A Novel Hybrid Model for Time-Series Forecasting and Frequency Pattern Analysis in Network Traffic](https://mlsanity.com/paper/2507.00304/) (2025)
- [Scaling Attention via Feature Sparsity](https://mlsanity.com/paper/2603.22300/) (2026)
- [Towards Identification and Intervention of Safety-Critical Parameters in Large Language Models](https://mlsanity.com/paper/2604.08297/) (2026)
- [Qrita: High-performance Top-k and Top-p using Pivot-based Truncation and Selection](https://mlsanity.com/paper/2602.01518/) (2026)
- [Stochastic Sparse Attention for Memory-Bound Inference](https://mlsanity.com/paper/2605.01910/) (2026)
- [A Faster Generalized Two-Stage Approximate Top-K](https://mlsanity.com/paper/2506.04165/) (2025)
- [Prof-K: Probabilistic One-Pass Filtering for Efficient Top-k Selection](https://mlsanity.com/paper/2608.12573/) (2026)

## Abstract (arXiv)

Top-k selection algorithms are fundamental in a wide range of applications, including high-performance computing, information retrieval, big data processing, and neural network model training. In this paper, we present RTop-K, a highly efficient parallel row-wise top-k selection algorithm specifically designed for GPUs. RTop-K leverages a binary search-based approach to optimize row-wise top-k selection, providing a scalable and accelerated solution. We conduct a detailed analysis of early stopping in our algorithm, showing that it effectively maintains the testing accuracy of neural network models while substantially improving performance. Our GPU implementation of RTop-K demonstrates superior performance over state-of-the-art row-wise top-k GPU implementations, achieving an average speed-up of up to 11.49× with early stopping and 7.29× without early stopping. Moreover, RTop-K accelerates the overall training workflow of MaxK-GNNs, delivering speed-ups ranging from 11.97% to 33.29% across different models and datasets.