# SCANN: Synthesis of Compact and Accurate Neural Networks

arXiv: https://arxiv.org/abs/1904.09090 · HTML: https://mlsanity.com/paper/1904.09090/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Apr 2019.

## Built on by (most cited first, 11 of 11)

- [STAIR: Learning Sparse Text and Image Representation in Grounded Tokens](https://mlsanity.com/paper/2301.13081/) (2023)
- [PhiNets: a scalable backbone for low-power AI at the edge](https://mlsanity.com/paper/2110.00337/) (2021)
- [DOCTOR: A Multi-Disease Detection Continual Learning Framework Based on Wearable Medical Sensors](https://mlsanity.com/paper/2305.05738/) (2023)
- [Efficient N:M Sparse DNN Training Using Algorithm, Architecture, and Dataflow Co-Design](https://mlsanity.com/paper/2309.13015/) (2023)
- [SHARKS: Smart Hacking Approaches for RisK Scanning in Internet-of-Things and Cyber-Physical Systems based on Machine Learning](https://mlsanity.com/paper/2101.02780/) (2021)
- [LSM-VEC: A Large-Scale Disk-Based System for Dynamic Vector Search](https://mlsanity.com/paper/2505.17152/) (2025)
- [Incremental Deep Neural Network Learning using Classification Confidence Thresholding](https://mlsanity.com/paper/2106.11437/) (2021)
- [Fast Design Space Exploration of Nonlinear Systems: Part I](https://mlsanity.com/paper/2104.01747/) (2021)
- [MHDeep: Mental Health Disorder Detection System based on Body-Area and Deep Neural Networks](https://mlsanity.com/paper/2102.10435/) (2021)
- [Compact Multi-level Sparse Neural Networks with Input Independent Dynamic Rerouting](https://mlsanity.com/paper/2112.10930/) (2021)
- [COMFORT: A Continual Fine-Tuning Framework for Foundation Models Targeted at Consumer Healthcare](https://mlsanity.com/paper/2409.09549/) (2024)

## Abstract (arXiv)

Deep neural networks (DNNs) have become the driving force behind recent artificial intelligence (AI) research. An important problem with implementing a neural network is the design of its architecture. Typically, such an architecture is obtained manually by exploring its hyperparameter space and kept fixed during training. This approach is time-consuming and inefficient. Another issue is that modern neural networks often contain millions of parameters, whereas many applications and devices require small inference models. However, efforts to migrate DNNs to such devices typically entail a significant loss of classification accuracy. To address these challenges, we propose a two-step neural network synthesis methodology, called DR+SCANN, that combines two complementary approaches to design compact and accurate DNNs. At the core of our framework is the SCANN methodology that uses three basic architecture-changing operations, namely connection growth, neuron growth, and connection pruning, to synthesize feed-forward architectures with arbitrary structure. SCANN encapsulates three synthesis methodologies that apply a repeated grow-and-prune paradigm to three architectural starting points. DR+SCANN combines the SCANN methodology with dataset dimensionality reduction to alleviate the curse of dimensionality. We demonstrate the efficacy of SCANN and DR+SCANN on various image and non-image datasets. We evaluate SCANN on MNIST and ImageNet benchmarks. In addition, we also evaluate the efficacy of using dimensionality reduction alongside SCANN (DR+SCANN) on nine small to medium-size datasets. We also show that our synthesis methodology yields neural networks that are much better at navigating the accuracy vs. energy efficiency space. This would enable neural network-based inference even on Internet-of-Things sensors.