# Dynamic Optimization of Neural Network Structures Using Probabilistic Modeling

arXiv: https://arxiv.org/abs/1801.07650 · HTML: https://mlsanity.com/paper/1801.07650/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Jan 2018.

## Built on by (most cited first, 4 of 4)

- [You Only Compress Once: Towards Effective and Elastic BERT Compression via Exploit-Explore Stochastic Nature Gradient](https://mlsanity.com/paper/2106.02435/) (2021)
- [Enhancing Weakly Supervised 3D Medical Image Segmentation through Probabilistic-aware Learning](https://mlsanity.com/paper/2403.02566/) (2024)
- [Efficient Search of Multiple Neural Architectures with Different Complexities via Importance Sampling](https://mlsanity.com/paper/2207.10334/) (2022)
- [A Novel Paradigm for Neural Computation: X-Net with Learnable Neurons and Adaptable Structure](https://mlsanity.com/paper/2401.01772/) (2024)

## Abstract (arXiv)

Deep neural networks (DNNs) are powerful machine learning models and have succeeded in various artificial intelligence tasks. Although various architectures and modules for the DNNs have been proposed, selecting and designing the appropriate network structure for a target problem is a challenging task. In this paper, we propose a method to simultaneously optimize the network structure and weight parameters during neural network training. We consider a probability distribution that generates network structures, and optimize the parameters of the distribution instead of directly optimizing the network structure. The proposed method can apply to the various network structure optimization problems under the same framework. We apply the proposed method to several structure optimization problems such as selection of layers, selection of unit types, and selection of connections using the MNIST, CIFAR-10, and CIFAR-100 datasets. The experimental results show that the proposed method can find the appropriate and competitive network structures.