# Spatially-sparse convolutional neural networks

arXiv: https://arxiv.org/abs/1409.6070 · HTML: https://mlsanity.com/paper/1409.6070/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Sep 2014.

## Built on by (most cited first, 30 of 33)

- [Delving into the Devils of Bird's-eye-view Perception: A Review, Evaluation and Recipe](https://mlsanity.com/paper/2209.05324/) (2022)
- [Semantics for Robotic Mapping, Perception and Interaction: A Survey](https://mlsanity.com/paper/2101.00443/) (2021)
- [Fully Convolutional One-Stage 3D Object Detection on LiDAR Range Images](https://mlsanity.com/paper/2205.13764/) (2022)
- [VISTA: Boosting 3D Object Detection via Dual Cross-VIew SpaTial Attention](https://mlsanity.com/paper/2203.09704/) (2022)
- [Spatial Pruned Sparse Convolution for Efficient 3D Object Detection](https://mlsanity.com/paper/2209.14201/) (2022)
- [PeRFception: Perception using Radiance Fields](https://mlsanity.com/paper/2208.11537/) (2022)
- [NeeDrop: Self-supervised Shape Representation from Sparse Point Clouds using Needle Dropping](https://mlsanity.com/paper/2111.15207/) (2021)
- [Traffic4cast at NeurIPS 2021 -- Temporal and Spatial Few-Shot Transfer Learning in Gridded Geo-Spatial Processes](https://mlsanity.com/paper/2203.17070/) (2022)
- [PCSCNet: Fast 3D Semantic Segmentation of LiDAR Point Cloud for Autonomous Car using Point Convolution and Sparse Convolution Network](https://mlsanity.com/paper/2202.10047/) (2022)
- [Improving Video Instance Segmentation via Temporal Pyramid Routing](https://mlsanity.com/paper/2107.13155/) (2021)
- [Memory-based Adapters for Online 3D Scene Perception](https://mlsanity.com/paper/2403.06974/) (2024)
- [SSF: Sparse Long-Range Scene Flow for Autonomous Driving](https://mlsanity.com/paper/2501.17821/) (2025)
- [Minkowski Tracker: A Sparse Spatio-Temporal R-CNN for Joint Object Detection and Tracking](https://mlsanity.com/paper/2208.10056/) (2022)
- [Sequential Point Clouds: A Survey](https://mlsanity.com/paper/2204.09337/) (2022)
- [Content-Adaptive Downsampling in Convolutional Neural Networks](https://mlsanity.com/paper/2305.09504/) (2023)
- [Optimized CNNs for Rapid 3D Point Cloud Object Recognition](https://mlsanity.com/paper/2412.02855/) (2024)
- [Which2comm: An Efficient Collaborative Perception Framework for 3D Object Detection](https://mlsanity.com/paper/2503.17175/) (2025)
- [Traffic4cast -- Large-scale Traffic Prediction using 3DResNet and Sparse-UNet](https://mlsanity.com/paper/2111.05990/) (2021)
- [Survey and Systematization of 3D Object Detection Models and Methods](https://mlsanity.com/paper/2201.09354/) (2022)
- [Learning Spatial and Temporal Variations for 4D Point Cloud Segmentation](https://mlsanity.com/paper/2207.04673/) (2022)
- [Putting 3D Spatially Sparse Networks on a Diet](https://mlsanity.com/paper/2112.01316/) (2021)
- [Exploiting Sparsity in Automotive Radar Object Detection Networks](https://mlsanity.com/paper/2308.07748/) (2023)
- [Sparse and Privacy-enhanced Representation for Human Pose Estimation](https://mlsanity.com/paper/2309.09515/) (2023)
- [DM3D: Distortion-Minimized Weight Pruning for Lossless 3D Object Detection](https://mlsanity.com/paper/2407.02098/) (2024)
- [On the Privacy Risks of Deploying Recurrent Neural Networks in Machine Learning Models](https://mlsanity.com/paper/2110.03054/) (2021)
- [SparseDet: Towards End-to-End 3D Object Detection](https://mlsanity.com/paper/2206.00960/) (2022)
- [CT Perfusion is All We Need: 4D CNN Segmentation of Penumbra and Core in Patients With Suspected Ischemic Stroke](https://mlsanity.com/paper/2303.08757/) (2023)
- [Binarizing Sparse Convolutional Networks for Efficient Point Cloud Analysis](https://mlsanity.com/paper/2303.15493/) (2023)
- [Exploring the Relationship: Transformative Adaptive Activation Functions in Comparison to Other Activation Functions](https://mlsanity.com/paper/2402.09249/) (2024)
- [Geometry Aware Field-to-field Transformations for 3D Semantic Segmentation](https://mlsanity.com/paper/2310.05133/) (2023)

## Abstract (arXiv)

Convolutional neural networks (CNNs) perform well on problems such as handwriting recognition and image classification. However, the performance of the networks is often limited by budget and time constraints, particularly when trying to train deep networks. Motivated by the problem of online handwriting recognition, we developed a CNN for processing spatially-sparse inputs; a character drawn with a one-pixel wide pen on a high resolution grid looks like a sparse matrix. Taking advantage of the sparsity allowed us more efficiently to train and test large, deep CNNs. On the CASIA-OLHWDB1.1 dataset containing 3755 character classes we get a test error of 3.82%. Although pictures are not sparse, they can be thought of as sparse by adding padding. Applying a deep convolutional network using sparsity has resulted in a substantial reduction in test error on the CIFAR small picture datasets: 6.28% on CIFAR-10 and 24.30% for CIFAR-100.