# Minuet: Accelerating 3D Sparse Convolutions on GPUs

arXiv: https://arxiv.org/abs/2401.06145 · HTML: https://mlsanity.com/paper/2401.06145/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 51 references that are in the map.
- First posted on arXiv 1 Dec 2023.

## Builds on (most cited first, 26 of 26)

- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [4D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networks](https://mlsanity.com/paper/1904.08755/) (2019)
- [Center-based 3D Object Detection and Tracking](https://mlsanity.com/paper/2006.11275/) (2020)
- [3D Semantic Segmentation with Submanifold Sparse Convolutional Networks](https://mlsanity.com/paper/1711.10275/) (2017)
- [Deep Learning for 3D Point Clouds: A Survey](https://mlsanity.com/paper/1912.12033/) (2019)
- [Searching Efficient 3D Architectures with Sparse Point-Voxel Convolution](https://mlsanity.com/paper/2007.16100/) (2020)
- [Point-Voxel CNN for Efficient 3D Deep Learning](https://mlsanity.com/paper/1907.03739/) (2019)
- [Voxel Transformer for 3D Object Detection](https://mlsanity.com/paper/2109.02497/) (2021)
- [Submanifold Sparse Convolutional Networks](https://mlsanity.com/paper/1706.01307/) (2017)
- [3D Object Detection with Pointformer](https://mlsanity.com/paper/2012.11409/) (2020)
- [OccuSeg: Occupancy-aware 3D Instance Segmentation](https://mlsanity.com/paper/2003.06537/) (2020)
- [Semantic3D.net: A new Large-scale Point Cloud Classification Benchmark](https://mlsanity.com/paper/1704.03847/) (2017)
- [Mix3D: Out-of-Context Data Augmentation for 3D Scenes](https://mlsanity.com/paper/2110.02210/) (2021)
- [TorchSparse: Efficient Point Cloud Inference Engine](https://mlsanity.com/paper/2204.10319/) (2022)
- [FlatFormer: Flattened Window Attention for Efficient Point Cloud Transformer](https://mlsanity.com/paper/2301.08739/) (2023)
- [LargeKernel3D: Scaling up Kernels in 3D Sparse CNNs](https://mlsanity.com/paper/2206.10555/) (2022)
- [SparseTIR: Composable Abstractions for Sparse Compilation in Deep Learning](https://mlsanity.com/paper/2207.04606/) (2022)
- [TensorIR: An Abstraction for Automatic Tensorized Program Optimization](https://mlsanity.com/paper/2207.04296/) (2022)
- [Hidet: Task-Mapping Programming Paradigm for Deep Learning Tensor Programs](https://mlsanity.com/paper/2210.09603/) (2022)
- [PointAcc: Efficient Point Cloud Accelerator](https://mlsanity.com/paper/2110.07600/) (2021)
- [Automatic Horizontal Fusion for GPU Kernels](https://mlsanity.com/paper/2007.01277/) (2020)
- [SemanticKITTI: A Dataset for Semantic Scene Understanding of LiDAR Sequences](https://mlsanity.com/paper/1904.01416/) (2019)
- [ASH: A Modern Framework for Parallel Spatial Hashing in 3D Perception](https://mlsanity.com/paper/2110.00511/) (2021)
- [3D-FCT: Simultaneous 3D Object Detection and Tracking Using Feature Correlation](https://mlsanity.com/paper/2110.02531/) (2021)

## Built on by (most cited first, 2 of 2)

- [SparseTem: Boosting the Efficiency of CNN-Based Video Encoders by Exploiting Temporal Continuity](https://mlsanity.com/paper/2410.20790/) (2024)
- [Spira: Exploiting Voxel Data Structural Properties for Efficient Sparse Convolution in Point Cloud Networks](https://mlsanity.com/paper/2511.20834/) (2025)

## Abstract (arXiv)

Sparse Convolution (SC) is widely used for processing 3D point clouds that are inherently sparse. Different from dense convolution, SC preserves the sparsity of the input point cloud by only allowing outputs to specific locations. To efficiently compute SC, prior SC engines first use hash tables to build a kernel map that stores the necessary General Matrix Multiplication (GEMM) operations to be executed (Map step), and then use a Gather-GEMM-Scatter process to execute these GEMM operations (GMaS step). In this work, we analyze the shortcomings of prior state-of-the-art SC engines, and propose Minuet, a novel memory-efficient SC engine tailored for modern GPUs. Minuet proposes to (i) replace the hash tables used in the Map step with a novel segmented sorting double-traversed binary search algorithm that highly utilizes the on-chip memory hierarchy of GPUs, (ii) use a lightweight scheme to autotune the tile size in the Gather and Scatter operations of the GMaS step, such that to adapt the execution to the particular characteristics of each SC layer, dataset, and GPU architecture, and (iii) employ a padding-efficient GEMM grouping approach that reduces both memory padding and kernel launching overheads. Our evaluations show that Minuet significantly outperforms prior SC engines by on average 1.74× (up to 2.22×) for end-to-end point cloud network executions. Our novel segmented sorting double-traversed binary search algorithm achieves superior speedups by 15.8× on average (up to 26.8×) over prior SC engines in the Map step. The source code of Minuet is publicly available at https://github.com/UofT-EcoSystem/Minuet.