# Astronomia ex machina: a history, primer, and outlook on neural networks in astronomy

arXiv: https://arxiv.org/abs/2211.03796 · HTML: https://mlsanity.com/paper/2211.03796/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 134 of its 349 references that are in the map.
- First posted on arXiv 7 Nov 2022.

## Builds on (most cited first, 30 of 134)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Flamingo: a Visual Language Model for Few-Shot Learning](https://mlsanity.com/paper/2204.14198/) (2022)

## Built on by (most cited first, 17 of 17)

- [Are Large Language Models a Threat to Digital Public Goods? Evidence from Activity on Stack Overflow](https://mlsanity.com/paper/2307.07367/) (2023)
- [EarthPT: a time series foundation model for Earth Observation](https://mlsanity.com/paper/2309.07207/) (2023)
- [AstroPT: Scaling Large Observation Models for Astronomy](https://mlsanity.com/paper/2405.14930/) (2024)
- [The Platonic Universe: Do Foundation Models See the Same Sky?](https://mlsanity.com/paper/2509.19453/) (2025)
- [A brief review of contrastive learning applied to astrophysics](https://mlsanity.com/paper/2306.05528/) (2023)
- [Efficient and Flexible Method for Reducing Moderate-size Deep Neural Networks with Condensation](https://mlsanity.com/paper/2405.01041/) (2024)
- [A review of unsupervised learning in astronomy](https://mlsanity.com/paper/2406.17316/) (2024)
- [Constructing sensible baselines for Integrated Gradients](https://mlsanity.com/paper/2412.13864/) (2024)
- [Beyond Scaling Curves: Internal Dynamics of Neural Networks Through the NTK Lens](https://mlsanity.com/paper/2507.05035/) (2025)
- [Discovering interpretable models of scientific image data with deep learning](https://mlsanity.com/paper/2402.03115/) (2024)
- [AstroSpy: On detecting Fake Images in Astronomy via Joint Image-Spectral Representations](https://mlsanity.com/paper/2407.06817/) (2024)
- [Dargana: fine-tuning EarthPT for dynamic tree canopy mapping from space](https://mlsanity.com/paper/2504.17321/) (2025)
- [OmniSpectra: A Unified Foundation Model for Native Resolution Astronomical Spectra](https://mlsanity.com/paper/2601.15351/) (2026)
- [Rapid prediction of lab-grown tissue properties using deep learning](https://mlsanity.com/paper/2303.18017/) (2023)
- [Datacube segmentation via Deep Spectral Clustering](https://mlsanity.com/paper/2401.17695/) (2024)
- [What AstroPT knows about galaxies, and what that can teach us about LLMs](https://mlsanity.com/paper/2608.22614/) (2026)
- [Gestalt: a meta-foundation model for astronomy](https://mlsanity.com/paper/2609.38312/) (2026)

## Abstract (arXiv)

In this review, we explore the historical development and future prospects of artificial intelligence (AI) and deep learning in astronomy. We trace the evolution of connectionism in astronomy through its three waves, from the early use of multilayer perceptrons, to the rise of convolutional and recurrent neural networks, and finally to the current era of unsupervised and generative deep learning methods. With the exponential growth of astronomical data, deep learning techniques offer an unprecedented opportunity to uncover valuable insights and tackle previously intractable problems. As we enter the anticipated fourth wave of astronomical connectionism, we argue for the adoption of GPT-like foundation models fine-tuned for astronomical applications. Such models could harness the wealth of high-quality, multimodal astronomical data to serve state-of-the-art downstream tasks. To keep pace with advancements driven by Big Tech, we propose a collaborative, open-source approach within the astronomy community to develop and maintain these foundation models, fostering a symbiotic relationship between AI and astronomy that capitalizes on the unique strengths of both fields.