# Constructing artificial life and materials scientists with accelerated AI using Deep AndersoNN

arXiv: https://arxiv.org/abs/2407.19724 · HTML: https://mlsanity.com/paper/2407.19724/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 31 references that are in the map.
- First posted on arXiv 29 Jul 2024.

## Builds on (most cited first, 12 of 12)

- [Deep Equilibrium Models](https://mlsanity.com/paper/1909.01377/) (2019)
- [Carbon Emissions and Large Neural Network Training](https://mlsanity.com/paper/2104.10350/) (2021)
- [Multiscale Deep Equilibrium Models](https://mlsanity.com/paper/2006.08656/) (2020)
- [Mindstorms in Natural Language-Based Societies of Mind](https://mlsanity.com/paper/2305.17066/) (2023)
- [JFB: Jacobian-Free Backpropagation for Implicit Networks](https://mlsanity.com/paper/2103.12803/) (2021)
- [On Training Implicit Models](https://mlsanity.com/paper/2111.05177/) (2021)
- [Stabilizing Equilibrium Models by Jacobian Regularization](https://mlsanity.com/paper/2106.14342/) (2021)
- [Pandemic Drugs at Pandemic Speed: Infrastructure for Accelerating COVID-19 Drug Discovery with Hybrid Machine Learning- and Physics-based Simulations on High Performance Computers](https://mlsanity.com/paper/2103.02843/) (2021)
- [Anderson Acceleration for Nonconvex ADMM Based on Douglas-Rachford Splitting](https://mlsanity.com/paper/2006.14539/) (2020)
- [IMPECCABLE: Integrated Modeling PipelinE for COVID Cure by Assessing Better LEads](https://mlsanity.com/paper/2010.06574/) (2020)
- [Targeting SARS-CoV-2 with AI- and HPC-enabled Lead Generation: A First Data Release](https://mlsanity.com/paper/2006.02431/) (2020)
- [Protein-Ligand Docking Surrogate Models: A SARS-CoV-2 Benchmark for Deep Learning Accelerated Virtual Screening](https://mlsanity.com/paper/2106.07036/) (2021)

## Abstract (arXiv)

Deep AndersoNN accelerates AI by exploiting the continuum limit as the number of explicit layers in a neural network approaches infinity and can be taken as a single implicit layer, known as a deep equilibrium model. Solving for deep equilibrium model parameters reduces to a nonlinear fixed point iteration problem, enabling the use of vector-to-vector iterative solvers and windowing techniques, such as Anderson extrapolation, for accelerating convergence to the fixed point deep equilibrium. Here we show that Deep AndersoNN achieves up to an order of magnitude of speed-up in training and inference. The method is demonstrated on density functional theory results for industrial applications by constructing artificial life and materials `scientists' capable of classifying drugs as strongly or weakly polar, metal-organic frameworks by pore size, and crystalline materials as metals, semiconductors, and insulators, using graph images of node-neighbor representations transformed from atom-bond networks. Results exhibit accuracy up to 98% and showcase synergy between Deep AndersoNN and machine learning capabilities of modern computing architectures, such as GPUs, for accelerated computational life and materials science by quickly identifying structure-property relationships. This paves the way for saving up to 90% of compute required for AI, reducing its carbon footprint by up to 60 gigatons per year by 2030, and scaling above memory limits of explicit neural networks in life and materials science, and beyond.