mlsanityData to 14 Oct 2026

A Study of Hybrid and Evolutionary Metaheuristics for Single Hidden Layer Feedforward Neural Network Architecture

arXiv 2506.15737 ↗First posted 17 Jun 2025cs.NE cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Training Artificial Neural Networks (ANNs) with Stochastic Gradient Descent (SGD) frequently encounters difficulties, including substantial computing expense and the risk of converging to local optima, attributable to its dependence on partial weight gradients. Therefore, this work investigates Particle Swarm Optimization (PSO) and Genetic Algorithms (GAs) - two population-based Metaheuristic Optimizers (MHOs) - as alternatives to SGD to mitigate these constraints. A hybrid PSO-SGD strategy is developed to improve local search efficiency. The findings indicate that the hybrid PSO-SGD technique decreases the median training MSE by 90 to 95 percent relative to conventional GA and PSO across various network sizes (e.g., from around 0.02 to approximately 0.001 in the Sphere function). RMHC attains substantial enhancements, reducing MSE by roughly 85 to 90 percent compared to GA. Simultaneously, RS consistently exhibits errors exceeding 0.3, signifying subpar performance. These findings underscore that hybrid and evolutionary procedures significantly improve training efficiency and accuracy compared to conventional optimization methods and imply that the Building Block Hypothesis (BBH) may still be valid, indicating that advantageous weight structures are retained during evolutionary search.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 17 of its 68 references that are in the map. First posted on arXiv 17 Jun 2025.

Builds on

17 of 68 references.

References · 17

  1. 2015Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networkscited by 2,054
  2. 2017Evolution Strategies as a Scalable Alternative to Reinforcement Learningcited by 574
  3. 2014Identifying and attacking the saddle point problem in high-dimensional non-convex optimizationcited by 289
  4. 2017Deep Neuroevolution: Genetic Algorithms Are a Competitive Alternative for Training Deep Neural Networks for Reinforcement Learningcited by 150
  5. 2013A Literature Survey of Benchmark Functions For Global Optimization Problemscited by 77
  6. 2019Increasing Generality in Machine Learning through Procedural Content Generationcited by 59
  7. 2024Convolutional neural network classification of cancer cytopathology images: taking breast cancer as an examplecited by 50
  8. 2024Bridging Evolutionary Algorithms and Reinforcement Learning: A Comprehensive Survey on Hybrid Algorithmscited by 26
  9. 2024Dealing with unbounded gradients in stochastic saddle-point optimizationcited by 16
  10. 2024A Survey on Universal Approximation Theoremscited by 15
  11. 2024Convergence rates for the Adam optimizercited by 15
  12. 2020A Survey on Activation Functions and their relation with Xavier and He Normal Initializationcited by 13
5 more
  1. 2024Coding schemes in neural networks learning classification taskscited by 11
  2. 2017Best practices for comparing optimization algorithmscited by 10
  3. 2021Parameterless Gene-pool Optimal Mixing Evolutionary Algorithmscited by 2
  4. 2024Efficient Distributed Training through Gradient Compression with Sparsification and Quantization Techniquescited by 2
  5. 2024Bridging the Gap Between Theory and Practice: Benchmarking Transfer Evolutionary Optimizationcited by 1

Built on by

Not cited yet.