mlsanityData to 7 Oct 2026

UDC: Unified DNAS for Compressible TinyML Models

arXiv 2201.05842 ↗First posted 15 Jan 2022cs.LGcited by 10

Not on the map. Why?

Abstract (arXiv)

Deploying TinyML models on low-cost IoT hardware is very challenging, due to limited device memory capacity. Neural processing unit (NPU) hardware address the memory challenge by using model compression to exploit weight quantization and sparsity to fit more parameters in the same footprint. However, designing compressible neural networks (NNs) is challenging, as it expands the design space across which we must make balanced trade-offs. This paper demonstrates Unified DNAS for Compressible (UDC) NNs, which explores a large search space to generate state-of-the-art compressible NNs for NPU. ImageNet results show UDC networks are up to 3.35× smaller (iso-accuracy) or 6.25% more accurate (iso-model size) than previous work.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 45 of its 87 references that are in the map. First posted on arXiv 15 Jan 2022.

Builds on

45 of 87 references.

References · 45

  1. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  2. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
  3. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,827
  4. 2016Wide Residual Networkscited by 3,057
  5. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,708
  6. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,291
  7. 2015Learning both Weights and Connections for Efficient Neural Networkscited by 1,998
  8. 2018DARTS: Differentiable Architecture Searchcited by 1,782
  9. 2018Efficient Neural Architecture Search via Parameter Sharingcited by 831
  10. 2019Once-for-All: Train One Network and Specialize it for Efficient Deploymentcited by 778
  11. 2017Channel Pruning for Accelerating Very Deep Neural Networkscited by 697
  12. 2017To prune, or not to prune: exploring the efficacy of pruning for model compressioncited by 596
33 more
  1. 2020What is the State of Neural Network Pruning?cited by 578
  2. 2018FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Searchcited by 480
  3. 2019Single Path One-Shot Neural Architecture Search with Uniform Samplingcited by 411
  4. 2016Accelerating the Super-Resolution Convolutional Neural Networkcited by 393
  5. 2017Lossy Image Compression with Compressive Autoencoderscited by 289
  6. 2020MCUNet: Tiny Deep Learning on IoT Devicescited by 269
  7. 2019Searching for A Robust Neural Architecture in Four GPU Hourscited by 258
  8. 2019Random Search and Reproducibility for Neural Architecture Searchcited by 253
  9. 2016Dynamic Network Surgery for Efficient DNNscited by 233
  10. 2017Incremental Network Quantization: Towards Lossless CNNs with Low-Precision Weightscited by 211
  11. 2020FBNetV2: Differentiable Neural Architecture Search for Spatial and Channel Dimensionscited by 152
  12. 2018Efficient Multi-objective Neural Architecture Search via Lamarckian Evolutioncited by 151
  13. 2020Soft Threshold Weight Reparameterization for Learnable Sparsitycited by 150
  14. 2020Training with Quantization Noise for Extreme Model Compressioncited by 145
  15. 2016Designing Energy-Efficient Convolutional Neural Networks using Energy-Aware Pruningcited by 133
  16. 2020Dynamic Model Pruning with Feedbackcited by 125
  17. 2021MLPerf Tiny Benchmarkcited by 118
  18. 2018Rethinking the Smaller-Norm-Less-Informative Assumption in Channel Pruning of Convolution Layerscited by 112
  19. 2020APQ: Joint Search for Network Architecture, Pruning and Quantization Policycited by 98
  20. 2018Practical Full Resolution Learned Lossless Image Compressioncited by 93
  21. 2021MCUNetV2: Memory-Efficient Patch-based Inference for Tiny Deep Learningcited by 87
  22. 2019Network Pruning via Transformable Architecture Searchcited by 80
  23. 2020DSNAS: Direct Neural Architecture Search without Parameter Retrainingcited by 69
  24. 2018Theoretical Analysis of Auto Rate-Tuning by Batch Normalizationcited by 65
  25. 2020μNAS: Constrained Neural Architecture Search for Microcontrollerscited by 54
  26. 2019SpArSe: Sparse Architecture Search for CNNs on Resource-Constrained Microcontrollerscited by 52
  27. 2020TinyLSTMs: Efficient Neural Speech Enhancement for Hearing Aidscited by 41
  28. 2020Rao-Blackwellizing the Straight-Through Gumbel-Softmax Gradient Estimatorcited by 37
  29. 2020Search What You Want: Barrier Panelty NAS for Mixed Precision Quantizationcited by 31
  30. 2020Journey Towards Tiny Perceptual Super-Resolutioncited by 27
  31. 2021Collapsible Linear Blocks for Super-Efficient Super Resolutioncited by 19
  32. 2019RC-DARTS: Resource Constrained Differentiable Architecture Searchcited by 13
  33. 2018Training wide residual networks for deployment using a single bit for each weightcited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 3; 2023: 4; 2024: 2; 2025: 0; 2026: 1.

10 papers.

  1. 2024Tiny Machine Learning: Progress and Futurescited by 40
  2. 2022Dynamic ConvNets on Tiny Devices via Nested Sparsitycited by 7
  3. 2023Enhancing Neural Architecture Search with Multiple Hardware Constraints for Deep Learning Model Deployment on Tiny IoT Devicescited by 7
  4. 2023PerfSAGE: Generalized Inference Performance Predictor for Arbitrary Deep Learning Models on Edge Devicescited by 4
  5. 2022Pex: Memory-efficient Microcontroller Deep Learning through Partial Executioncited by 3
  6. 2023Rankitect: Ranking Architecture Search Battling World-class Engineers at Meta Scalecited by 3
  7. 2024MCUBERT: Memory-Efficient BERT Inference on Commodity Microcontrollerscited by 3
  8. 2022Multi-Complexity-Loss DNAS for Energy-Efficient and Memory-Constrained Deep Neural Networkscited by 2
  9. 2023SiGeo: Sub-One-Shot NAS via Information Theory and Geometry of Loss Landscapecited by 1
  10. 2026MobileLLM-Flash: Latency-Guided On-Device LLM Design for Industry Scale Deploymentcited by 1