mlsanityData to 7 Oct 2026

Based on What We Can Control Artificial Neural Networks

arXiv 2310.05692 ↗First posted 9 Oct 2023cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

How can the stability and efficiency of Artificial Neural Networks (ANNs) be ensured through a systematic analysis method? This paper seeks to address that query. While numerous factors can influence the learning process of ANNs, utilizing knowledge from control systems allows us to analyze its system function and simulate system responses. Although the complexity of most ANNs is extremely high, we still can analyze each factor (e.g., optimiser, hyperparameters) by simulating their system response. This new method also can potentially benefit the development of new optimiser and learning system, especially when discerning which components adversely affect ANNs. Controlling ANNs can benefit from the design of optimiser and learning system, as (1) all optimisers act as controllers, (2) all learning systems operate as control systems with inputs and outputs, and (3) the optimiser should match the learning system. Please find codes: urlhttps://github.com/RandomUserName2023/Control-ANNs.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 20 of its 44 references that are in the map. First posted on arXiv 9 Oct 2023.

Builds on

20 of 44 references.

References · 20

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  5. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  6. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  7. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  8. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  9. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  10. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  11. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
  12. 2016Identity Mappings in Deep Residual Networkscited by 1,960
8 more
  1. 2022The Forward-Forward Algorithm: Some Preliminary Investigationscited by 262
  2. 2020An Improved Analysis of Stochastic Gradient Descent with Momentumcited by 117
  3. 2017On the convergence properties of a K-step averaging stochastic gradient descent algorithm for nonconvex optimizationcited by 91
  4. 2022Low-Pass Filtering SGD for Recovering Flat Optima in the Deep Learning Optimization Landscapecited by 39
  5. 2020A Second look at Exponential and Cosine Step Sizes: Simplicity, Adaptivity, and Performancecited by 26
  6. 2018An improvement of the convergence proof of the ADAM-Optimizercited by 19
  7. 2017DenseNet for Dense Flowcited by 15
  8. 2015Max-Pooling Dropout for Regularization of Convolutional Neural Networkscited by 11

Built on by

Not cited yet.