mlsanityData to 7 Oct 2026

Dynamic Neural Network Architectural and Topological Adaptation and Related Methods -- A Survey

arXiv 2108.10066 ↗First posted 28 Jul 2021cs.LG cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Training and inference in deep neural networks (DNNs) has, due to a steady increase in architectural complexity and data set size, lead to the development of strategies for reducing time and space requirements of DNN training and inference, which is of particular importance in scenarios where training takes place in resource constrained computation environments or inference is part of a time critical application. In this survey, we aim to provide a general overview and categorization of state-of-the-art (SOTA) of techniques to reduced DNN training and inference time and space complexities with a particular focus on architectural adaptions.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 50 of its 84 references that are in the map. First posted on arXiv 28 Jul 2021.

Builds on

50 of 84 references.

References · 50

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  3. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  4. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  5. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  6. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  7. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
  8. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,827
  9. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,590
  10. 2020Supervised Contrastive Learningcited by 2,803
  11. 2016Aggregated Residual Transformations for Deep Neural Networkscited by 2,540
  12. 2016Xception: Deep Learning with Depthwise Separable Convolutionscited by 2,320
38 more
  1. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,291
  2. 2019Searching for MobileNetV3cited by 1,984
  3. 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,548
  4. 2017ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devicescited by 1,194
  5. 2018ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Designcited by 1,132
  6. 2016SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model sizecited by 1,054
  7. 2017GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networkscited by 816
  8. 2017Self-Normalizing Neural Networkscited by 591
  9. 2021Carbon Emissions and Large Neural Network Trainingcited by 561
  10. 2016Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networkscited by 541
  11. 2015Variational Dropout and the Local Reparameterization Trickcited by 489
  12. 2018LQ-Nets: Learned Quantization for Highly Accurate and Compact Deep Neural Networkscited by 272
  13. 2019A Survey of the Recent Architectures of Deep Convolutional Neural Networkscited by 211
  14. 2020Carbontracker: Tracking and Predicting the Carbon Footprint of Training Deep Learning Modelscited by 168
  15. 2017Interpretable Convolutional Neural Networkscited by 161
  16. 2018Visual Interpretability for Deep Learning: a Surveycited by 155
  17. 2018Demystifying Parallel and Distributed Deep Learning: An In-Depth Concurrency Analysiscited by 154
  18. 2019Quantization Networkscited by 151
  19. 2016QSGD: Communication-Efficient SGD via Gradient Quantization and Encodingcited by 147
  20. 2018How to Start Training: The Effect of Initialization and Architecturecited by 109
  21. 2019Bit-Flip Attack: Crushing Neural Network with Progressive Bit Searchcited by 106
  22. 2017MorphNet: Fast & Simple Resource-Constrained Structure Learning of Deep Networkscited by 92
  23. 2018Channel Gating Neural Networkscited by 80
  24. 2018A Survey on Methods and Theories of Quantized Neural Networkscited by 75
  25. 2020Adversarial Robustness on In- and Out-Distribution Improves Explainabilitycited by 73
  26. 2019Terminal Brain Damage: Exposing the Graceless Degradation in Deep Neural Networks Under Hardware Fault Attackscited by 55
  27. 2017Structured Bayesian Pruning via Log-Normal Multiplicative Noisecited by 39
  28. 2018A Quantization-Friendly Separable Convolution for MobileNetscited by 22
  29. 2019QPyTorch: A Low-Precision Arithmetic Simulation Frameworkcited by 21
  30. 2020Procrustes: a Dataflow and Accelerator for Sparse Deep Neural Network Trainingcited by 18
  31. 2018Deep Learning Towards Mobile Applicationscited by 16
  32. 2019Fully Learnable Group Convolution for Acceleration of Deep Neural Networkscited by 16
  33. 2020FNA++: Fast Network Adaptation via Parameter Remapping and Architecture Searchcited by 16
  34. 2020Gradient Amplification: An efficient way to train deep neural networkscited by 15
  35. 2018Full deep neural network training on a pruned weight budgetcited by 11
  36. 2020Multi-Precision Policy Enforced Training (MuPPET): A precision-switching strategy for quantised fixed-point training of CNNscited by 11
  37. 2017TensorQuant - A Simulation Toolbox for Deep Neural Network Quantizationcited by 8
  38. 2017An Enhanced Hybrid MobileNetcited by 5

Built on by

Not cited yet.