# TorchSparse++: Efficient Training and Inference Framework for Sparse Convolution on GPUs

arXiv: https://arxiv.org/abs/2311.12862 · HTML: https://mlsanity.com/paper/2311.12862/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 55 references that are in the map.
- First posted on arXiv 25 Oct 2023.

## Builds on (most cited first, 30 of 33)

- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [Scalability in Perception for Autonomous Driving: Waymo Open Dataset](https://mlsanity.com/paper/1912.04838/) (2019)
- [Fast Graph Representation Learning with PyTorch Geometric](https://mlsanity.com/paper/1903.02428/) (2019)
- [PointPillars: Fast Encoders for Object Detection from Point Clouds](https://mlsanity.com/paper/1812.05784/) (2018)
- [Modeling Relational Data with Graph Convolutional Networks](https://mlsanity.com/paper/1703.06103/) (2017)
- [4D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networks](https://mlsanity.com/paper/1904.08755/) (2019)
- [3D Semantic Segmentation with Submanifold Sparse Convolutional Networks](https://mlsanity.com/paper/1711.10275/) (2017)
- [TransFusion: Robust LiDAR-Camera Fusion for 3D Object Detection with Transformers](https://mlsanity.com/paper/2203.11496/) (2022)
- [Searching Efficient 3D Architectures with Sparse Point-Voxel Convolution](https://mlsanity.com/paper/2007.16100/) (2020)
- [Caffe: Convolutional Architecture for Fast Feature Embedding](https://mlsanity.com/paper/1408.5093/) (2014)
- [(AF)2-S3Net: Attentive Feature Fusion with Adaptive Feature Selection for Sparse Semantic Segmentation Network](https://mlsanity.com/paper/2102.04530/) (2021)
- [Offboard 3D Object Detection from Point Cloud Sequences](https://mlsanity.com/paper/2103.05073/) (2021)
- [DSVT: Dynamic Sparse Voxel Transformer with Rotated Sets](https://mlsanity.com/paper/2301.06051/) (2023)
- [S3CNet: A Sparse Semantic Scene Completion Network for LiDAR Point Clouds](https://mlsanity.com/paper/2012.09242/) (2020)
- [TorchSparse: Efficient Point Cloud Inference Engine](https://mlsanity.com/paper/2204.10319/) (2022)
- [FlatFormer: Flattened Window Attention for Efficient Point Cloud Transformer](https://mlsanity.com/paper/2301.08739/) (2023)
- [MPPNet: Multi-Frame Feature Intertwining with Proxy Points for 3D Temporal Object Detection](https://mlsanity.com/paper/2205.05979/) (2022)
- [Not All Pixels Are Equal: Difficulty-aware Semantic Segmentation via Deep Layer Cascade](https://mlsanity.com/paper/1704.01344/) (2017)
- [Green Hierarchical Vision Transformer for Masked Image Modeling](https://mlsanity.com/paper/2205.13515/) (2022)
- [SpArch: Efficient Architecture for Sparse Matrix Multiplication](https://mlsanity.com/paper/2002.08947/) (2020)
- [SparseTIR: Composable Abstractions for Sparse Compilation in Deep Learning](https://mlsanity.com/paper/2207.04606/) (2022)
- [Dual-side Sparse Tensor Core](https://mlsanity.com/paper/2105.09564/) (2021)
- [PointAcc: Efficient Point Cloud Accelerator](https://mlsanity.com/paper/2110.07600/) (2021)
- [Efficient and Robust LiDAR-Based End-to-End Navigation](https://mlsanity.com/paper/2105.09932/) (2021)
- [Nimble: Efficiently Compiling Dynamic Neural Networks for Model Inference](https://mlsanity.com/paper/2006.03031/) (2020)
- [Bolt: Bridging the Gap between Auto-tuners and Hardware-native Performance](https://mlsanity.com/paper/2110.15238/) (2021)
- [An Efficient Hardware Accelerator for Structured Sparse Convolutional Neural Networks on FPGAs](https://mlsanity.com/paper/2001.01955/) (2020)
- [PVNAS: 3D Neural Architecture Search with Point-Voxel Convolution](https://mlsanity.com/paper/2204.11797/) (2022)
- [Recurrent Residual Module for Fast Inference in Videos](https://mlsanity.com/paper/1802.09723/) (2018)

## Built on by (most cited first, 30 of 36)

- [One-2-3-45++: Fast Single Image to 3D Objects with Consistent Multi-View Generation and 3D Diffusion](https://mlsanity.com/paper/2311.07885/) (2023)
- [Native and Compact Structured Latents for 3D Generation](https://mlsanity.com/paper/2512.14692/) (2025)
- [DistriFusion: Distributed Parallel Inference for High-Resolution Diffusion Models](https://mlsanity.com/paper/2402.19481/) (2024)
- [MeshFormer: High-Quality Mesh Generation with 3D-Guided Reconstruction Model](https://mlsanity.com/paper/2408.10198/) (2024)
- [G3R: Gradient Guided Generalizable Reconstruction](https://mlsanity.com/paper/2409.19405/) (2024)
- [fVDB: A Deep-Learning Framework for Sparse, Large-Scale, and High-Performance Spatial Intelligence](https://mlsanity.com/paper/2407.01781/) (2024)
- [LiMoE: Mixture of LiDAR Representation Learners from Automotive Scenes](https://mlsanity.com/paper/2501.04004/) (2025)
- [LidaRF: Delving into Lidar for Neural Radiance Field on Street Scenes](https://mlsanity.com/paper/2405.00900/) (2024)
- [Training-Free Adaptive Diffusion with Bounded Difference Approximation Strategy](https://mlsanity.com/paper/2410.09873/) (2024)
- [RENO: Real-Time Neural Compression for 3D LiDAR Point Clouds](https://mlsanity.com/paper/2503.12382/) (2025)
- [3EED: Ground Everything Everywhere in 3D](https://mlsanity.com/paper/2511.01755/) (2025)
- [Calib3D: Calibrating Model Preferences for Reliable 3D Scene Understanding](https://mlsanity.com/paper/2403.17010/) (2024)
- [SPIRAL: Semantic-Aware Progressive LiDAR Scene Generation and Understanding](https://mlsanity.com/paper/2505.22643/) (2025)
- [Flux4D: Flow-based Unsupervised 4D Reconstruction](https://mlsanity.com/paper/2512.03210/) (2025)
- [Human-in-the-Loop Local Corrections of 3D Scene Layouts via Infilling](https://mlsanity.com/paper/2503.11806/) (2025)
- [Sparse Refinement for Efficient High-Resolution Semantic Segmentation](https://mlsanity.com/paper/2407.19014/) (2024)
- [Fast Feature Field (F3): A Predictive Representation of Events](https://mlsanity.com/paper/2509.25146/) (2025)
- [DM3D: Distortion-Minimized Weight Pruning for Lossless 3D Object Detection](https://mlsanity.com/paper/2407.02098/) (2024)
- [Re-Densification Meets Cross-Scale Propagation: Real-Time Neural Compression of LiDAR Point Clouds](https://mlsanity.com/paper/2508.20466/) (2025)
- [EZ-SP: Fast and Lightweight Superpoint-Based 3D Segmentation](https://mlsanity.com/paper/2512.00385/) (2025)
- [Input-Dependent Power Usage in GPUs](https://mlsanity.com/paper/2409.18324/) (2024)
- [Towards scalable organ level 3D plant segmentation: Bridging the data algorithm computing gap](https://mlsanity.com/paper/2509.06329/) (2025)
- [AnyPcc: Compressing Any Point Cloud with a Single Universal Model](https://mlsanity.com/paper/2510.20331/) (2025)
- [Towards Practical Lossless Neural Compression for LiDAR Point Clouds](https://mlsanity.com/paper/2603.25260/) (2026)
- [Sparse Representation Learning for Vessels](https://mlsanity.com/paper/2605.01382/) (2026)
- [Deep Microcompression: Structured Pruning and Bit-packed Quantization for Microcontrollers](https://mlsanity.com/paper/2609.05081/) (2026)
- [MediSwift: Efficient Sparse Pre-trained Biomedical Language Models](https://mlsanity.com/paper/2403.00952/) (2024)
- [Generalizable Neural Reconstruction of High-Fidelity Surfaces via Sparse Volumetric Representations](https://mlsanity.com/paper/2507.05952/) (2025)
- [SPLite Hand: Sparsity-Aware Lightweight 3D Hand Pose Estimation](https://mlsanity.com/paper/2510.16396/) (2025)
- [ELiC: Efficient LiDAR Geometry Compression via Cross-Bit-depth Feature Propagation and Bag-of-Encoders](https://mlsanity.com/paper/2511.14070/) (2025)

## Abstract (arXiv)

Sparse convolution plays a pivotal role in emerging workloads, including point cloud processing in AR/VR, autonomous driving, and graph understanding in recommendation systems. Since the computation pattern is sparse and irregular, specialized high-performance kernels are required. Existing GPU libraries offer two dataflow types for sparse convolution. The gather-GEMM-scatter dataflow is easy to implement but not optimal in performance, while the dataflows with overlapped computation and memory access (e.g.implicit GEMM) are highly performant but have very high engineering costs. In this paper, we introduce TorchSparse++, a new GPU library that achieves the best of both worlds. We create a highly efficient Sparse Kernel Generator that generates performant sparse convolution kernels at less than one-tenth of the engineering cost of the current state-of-the-art system. On top of this, we design the Sparse Autotuner, which extends the design space of existing sparse convolution libraries and searches for the best dataflow configurations for training and inference workloads. Consequently, TorchSparse++ achieves 2.9x, 3.3x, 2.2x and 1.7x measured end-to-end speedup on an NVIDIA A100 GPU over state-of-the-art MinkowskiEngine, SpConv 1.2, TorchSparse and SpConv v2 in inference; and is 1.2-1.3x faster than SpConv v2 in mixed precision training across seven representative autonomous driving benchmarks. It also seamlessly supports graph convolutions, achieving 2.6-7.6x faster inference speed compared with state-of-the-art graph deep learning libraries.