# Towards Galaxy Foundation Models with Hybrid Contrastive Learning

arXiv: https://arxiv.org/abs/2206.11927 · HTML: https://mlsanity.com/paper/2206.11927/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 28 references that are in the map.
- First posted on arXiv 23 Jun 2022.

## Builds on (most cited first, 16 of 16)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [Supervised Contrastive Learning](https://mlsanity.com/paper/2004.11362/) (2020)
- [What Makes for Good Views for Contrastive Learning?](https://mlsanity.com/paper/2005.10243/) (2020)
- [Using Self-Supervised Learning Can Improve Model Robustness and Uncertainty](https://mlsanity.com/paper/1906.12340/) (2019)
- [Keystroke Dynamics as Part of Lifelogging](https://mlsanity.com/paper/2010.10144/) (2020)
- [Big Self-Supervised Models Advance Medical Image Classification](https://mlsanity.com/paper/2101.05224/) (2021)
- [Self-supervised Pretraining of Visual Features in the Wild](https://mlsanity.com/paper/2103.01988/) (2021)
- [Contrastive Learning based Hybrid Networks for Long-Tailed Image Classification](https://mlsanity.com/paper/2103.14267/) (2021)
- [Contrastive Learning for Label-Efficient Semantic Segmentation](https://mlsanity.com/paper/2012.06985/) (2020)
- [Rotation-invariant convolutional neural networks for galaxy morphology prediction](https://mlsanity.com/paper/1503.07077/) (2015)
- [Self-Supervised Representation Learning for Astronomical Images](https://mlsanity.com/paper/2012.13083/) (2020)
- [Practical Galaxy Morphology Tools from Deep Supervised Representation Learning](https://mlsanity.com/paper/2110.12735/) (2021)
- [Mining for Strong Gravitational Lenses with Self-supervised Learning](https://mlsanity.com/paper/2110.00023/) (2021)

## Built on by (most cited first, 10 of 10)

- [Masked Particle Modeling on Sets: Towards Self-Supervised High Energy Physics Foundation Models](https://mlsanity.com/paper/2401.13537/) (2024)
- [Re-Simulation-based Self-Supervised Learning for Pre-Training Foundation Models](https://mlsanity.com/paper/2403.07066/) (2024)
- [Astronomia ex machina: a history, primer, and outlook on neural networks in astronomy](https://mlsanity.com/paper/2211.03796/) (2022)
- [DeepAstroUDA: Semi-Supervised Universal Domain Adaptation for Cross-Survey Galaxy Morphology Classification and Anomaly Detection](https://mlsanity.com/paper/2302.02005/) (2023)
- [A brief review of contrastive learning applied to astrophysics](https://mlsanity.com/paper/2306.05528/) (2023)
- [Scaling Laws for Galaxy Images](https://mlsanity.com/paper/2404.02973/) (2024)
- [Are vision language models robust to uncertain inputs?](https://mlsanity.com/paper/2505.11804/) (2025)
- [AstroCLIP: A Cross-Modal Foundation Model for Galaxies](https://mlsanity.com/paper/2310.03024/) (2023)
- [Rare Galaxy Classes Identified In Foundation Model Representations](https://mlsanity.com/paper/2312.02910/) (2023)
- [Deep Learning Segmentation of Spiral Arms and Bars](https://mlsanity.com/paper/2312.02908/) (2023)

## Abstract (arXiv)

New astronomical tasks are often related to earlier tasks for which labels have already been collected. We adapt the contrastive framework BYOL to leverage those labels as a pretraining task while also enforcing augmentation invariance. For large-scale pretraining, we introduce GZ-Evo v0.1, a set of 96.5M volunteer responses for 552k galaxy images plus a further 1.34M comparable unlabelled galaxies. Most of the 206 GZ-Evo answers are unknown for any given galaxy, and so our pretraining task uses a Dirichlet loss that naturally handles unknown answers. GZ-Evo pretraining, with or without hybrid learning, improves on direct training even with plentiful downstream labels (+4% accuracy with 44k labels). Our hybrid pretraining/contrastive method further improves downstream accuracy vs. pretraining or contrastive learning, especially in the low-label transfer regime (+6% accuracy with 750 labels).