# MorphNet: Fast & Simple Resource-Constrained Structure Learning of Deep Networks

arXiv: https://arxiv.org/abs/1711.06798 · HTML: https://mlsanity.com/paper/1711.06798/

## Key facts

- Cited by 94 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Nov 2017.

## Built on by (most cited first, 30 of 94)

- [Pruning and Quantization for Deep Neural Network Acceleration: A Survey](https://mlsanity.com/paper/2101.09671/) (2021)
- [A Survey on Deep Neural Network Pruning-Taxonomy, Comparison, Analysis, and Recommendations](https://mlsanity.com/paper/2308.06767/) (2023)
- [A Comprehensive Survey on Hardware-Aware Neural Architecture Search](https://mlsanity.com/paper/2101.09336/) (2021)
- [DECORE: Deep Compression with Reinforcement Learning](https://mlsanity.com/paper/2106.06091/) (2021)
- [Computationally-Efficient Neural Image Compression with Shallow Decoders](https://mlsanity.com/paper/2304.06244/) (2023)
- [ChipNet: Budget-Aware Pruning with Heaviside Continuous Approximations](https://mlsanity.com/paper/2102.07156/) (2021)
- [Learned Gradient Compression for Distributed Deep Learning](https://mlsanity.com/paper/2103.08870/) (2021)
- [Layer-wise Model Pruning based on Mutual Information](https://mlsanity.com/paper/2108.12594/) (2021)
- [Knowledge Evolution in Neural Networks](https://mlsanity.com/paper/2103.05152/) (2021)
- [Low-Light Image and Video Enhancement: A Comprehensive Survey and Beyond](https://mlsanity.com/paper/2212.10772/) (2022)
- [Differentiable Network Pruning for Microcontrollers](https://mlsanity.com/paper/2110.08350/) (2021)
- [Lightweight Neural Architecture Search for Temporal Convolutional Networks at the Edge](https://mlsanity.com/paper/2301.10281/) (2023)
- [Q-PPG: Energy-Efficient PPG-based Heart Rate Monitoring on Wearable Devices](https://mlsanity.com/paper/2203.14907/) (2022)
- [NetAdaptV2: Efficient Neural Architecture Search with Fast Super-Network Training and Architecture Optimization](https://mlsanity.com/paper/2104.00031/) (2021)
- [LANA: Latency Aware Network Acceleration](https://mlsanity.com/paper/2107.10624/) (2021)
- [DyRep: Bootstrapping Training with Dynamic Re-parameterization](https://mlsanity.com/paper/2203.12868/) (2022)
- [Dissecting FLOPs along input dimensions for GreenAI cost estimations](https://mlsanity.com/paper/2107.11949/) (2021)
- [Resource-Efficient Deep Learning: A Survey on Model-, Arithmetic-, and Implementation-Level Techniques](https://mlsanity.com/paper/2112.15131/) (2021)
- [PLiNIO: A User-Friendly Library of Gradient-based Methods for Complexity-aware DNN Optimization](https://mlsanity.com/paper/2307.09488/) (2023)
- [Growing Tiny Networks: Spotting Expressivity Bottlenecks and Fixing Them Optimally](https://mlsanity.com/paper/2405.19816/) (2024)
- [FlashSVD: Memory-Efficient Inference with Streaming for Low-Rank Models](https://mlsanity.com/paper/2508.01506/) (2025)
- [Adaptive Precision Training (AdaPT): A dynamic fixed point quantized training approach for DNNs](https://mlsanity.com/paper/2107.13490/) (2021)
- [Dynamic ConvNets on Tiny Devices via Nested Sparsity](https://mlsanity.com/paper/2203.03324/) (2022)
- [Embedding Temporal Convolutional Networks for Energy-Efficient PPG-Based Heart Rate Monitoring](https://mlsanity.com/paper/2203.04396/) (2022)
- [Designing and Training of Lightweight Neural Networks on Edge Devices using Early Halting in Knowledge Distillation](https://mlsanity.com/paper/2209.15560/) (2022)
- [Enhancing Neural Architecture Search with Multiple Hardware Constraints for Deep Learning Model Deployment on Tiny IoT Devices](https://mlsanity.com/paper/2310.07217/) (2023)
- [Shaving Weights with Occam's Razor: Bayesian Sparsification for Neural Networks Using the Marginal Likelihood](https://mlsanity.com/paper/2402.15978/) (2024)
- [Update Compression for Deep Neural Networks on the Edge](https://mlsanity.com/paper/2203.04516/) (2022)
- [Robust and Energy-efficient PPG-based Heart-Rate Monitoring](https://mlsanity.com/paper/2203.16339/) (2022)
- [Human Activity Recognition on Microcontrollers with Quantized and Adaptive Deep Neural Networks](https://mlsanity.com/paper/2209.00839/) (2022)

## Abstract (arXiv)

We present MorphNet, an approach to automate the design of neural network structures. MorphNet iteratively shrinks and expands a network, shrinking via a resource-weighted sparsifying regularizer on activations and expanding via a uniform multiplicative factor on all layers. In contrast to previous approaches, our method is scalable to large networks, adaptable to specific resource constraints (e.g. the number of floating-point operations per inference), and capable of increasing the network's performance. When applied to standard network architectures on a wide variety of datasets, our approach discovers novel structures in each domain, obtaining higher performance while respecting the resource constraint.