# Neural Networks for Generating Better Local Optima in Topology Optimization

arXiv: https://arxiv.org/abs/2407.17957 · HTML: https://mlsanity.com/paper/2407.17957/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 112 references that are in the map.
- First posted on arXiv 25 Jul 2024.

## Builds on (most cited first, 30 of 33)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators](https://mlsanity.com/paper/1910.03193/) (2019)
- [How transferable are features in deep neural networks?](https://mlsanity.com/paper/1411.1792/) (2014)
- [Automatic differentiation in machine learning: a survey](https://mlsanity.com/paper/1502.05767/) (2015)
- [Why gradient clipping accelerates training: A theoretical justification for adaptivity](https://mlsanity.com/paper/1905.11881/) (2019)
- [Model Reduction and Neural Networks for Parametric PDEs](https://mlsanity.com/paper/2005.03180/) (2020)
- [Deep Learning Methods for Reynolds-Averaged Navier-Stokes Simulations of Airfoil Flows](https://mlsanity.com/paper/1810.08217/) (2018)
- [Polyconvex anisotropic hyperelasticity with neural networks](https://mlsanity.com/paper/2106.14623/) (2021)
- [Unsupervised discovery of interpretable hyperelastic constitutive laws](https://mlsanity.com/paper/2010.13496/) (2020)
- [Inverse-design of nonlinear mechanical metamaterials via video denoising diffusion models](https://mlsanity.com/paper/2305.19836/) (2023)
- [Neural networks meet hyperelasticity: A guide to enforcing physics](https://mlsanity.com/paper/2302.02403/) (2023)
- [Neural reparameterization improves structural optimization](https://mlsanity.com/paper/1909.04240/) (2019)
- [Deep learning for determining a near-optimal topological design without any iteration](https://mlsanity.com/paper/1801.05463/) (2018)
- [On the use of Artificial Neural Networks in Topology Optimisation](https://mlsanity.com/paper/2208.02563/) (2022)
- [Stress Field Prediction in Cantilevered Structures Using Convolutional Neural Networks](https://mlsanity.com/paper/1808.08914/) (2018)
- [A Comprehensive Survey of Deep Transfer Learning for Anomaly Detection in Industrial Time Series: Methods, Applications, and Directions](https://mlsanity.com/paper/2307.05638/) (2023)
- [Deep Learning in Deterministic Computational Mechanics](https://mlsanity.com/paper/2309.15421/) (2023)
- [A Deep Neural Network Surrogate Modeling Benchmark for Temperature Field Prediction of Heat Source Layout](https://mlsanity.com/paper/2103.11177/) (2021)
- [Real-Time Topology Optimization in 3D via Deep Transfer Learning](https://mlsanity.com/paper/2102.07657/) (2021)
- [Neural topology optimization: the good, the bad, and the ugly](https://mlsanity.com/paper/2407.13954/) (2024)
- [Quantitative reconstruction of defects in multi-layered bonded composites using fully convolutional network-based ultrasonic inversion](https://mlsanity.com/paper/2109.07284/) (2021)
- [On the Use of Neural Networks for Full Waveform Inversion](https://mlsanity.com/paper/2303.03260/) (2023)
- [Transfer Learning Enhanced Full Waveform Inversion](https://mlsanity.com/paper/2302.11259/) (2023)

## Built on by (most cited first, 4 of 4)

- [A surrogate model for topology optimisation of elastic structures via parametric autoencoders](https://mlsanity.com/paper/2507.22539/) (2025)
- [Accelerating Full Waveform Inversion By Transfer Learning](https://mlsanity.com/paper/2408.00695/) (2024)
- [Neural surrogates for designing gravitational wave detectors](https://mlsanity.com/paper/2511.19364/) (2025)
- [How the Classical Baseline Shapes Machine-Learning Speed Claims in Topology Optimization](https://mlsanity.com/paper/2610.16051/) (2026)

## Abstract (arXiv)

Neural networks have recently been employed as material discretizations within adjoint optimization frameworks for inverse problems and topology optimization. While advantageous regularization effects and better optima have been found for some inverse problems, the benefit for topology optimization has been limited -- where the focus of investigations has been the compliance problem. We demonstrate how neural network material discretizations can, under certain conditions, find better local optima in more challenging optimization problems, where we here specifically consider acoustic topology optimization. The chances of identifying a better optimum can significantly be improved by running multiple partial optimizations with different neural network initializations. Furthermore, we show that the neural network material discretization's advantage comes from the interplay with the Adam optimizer and emphasize its current limitations when competing with constrained and higher-order optimization techniques. At the moment, this discretization has only been shown to be beneficial for unconstrained first-order optimization.