# Self-Supervised Representation Learning for Astronomical Images

arXiv: https://arxiv.org/abs/2012.13083 · HTML: https://mlsanity.com/paper/2012.13083/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Dec 2020.

## Built on by (most cited first, 18 of 18)

- [Astronomia ex machina: a history, primer, and outlook on neural networks in astronomy](https://mlsanity.com/paper/2211.03796/) (2022)
- [PAPERCLIP: Associating Astronomical Observations and Natural Language with Multi-Modal Models](https://mlsanity.com/paper/2403.08851/) (2024)
- [Self-supervised similarity search for large scientific datasets](https://mlsanity.com/paper/2110.13151/) (2021)
- [Towards Galaxy Foundation Models with Hybrid Contrastive Learning](https://mlsanity.com/paper/2206.11927/) (2022)
- [AstroPT: Scaling Large Observation Models for Astronomy](https://mlsanity.com/paper/2405.14930/) (2024)
- [Mining for Strong Gravitational Lenses with Self-supervised Learning](https://mlsanity.com/paper/2110.00023/) (2021)
- [A brief review of contrastive learning applied to astrophysics](https://mlsanity.com/paper/2306.05528/) (2023)
- [Neural Network Emulator for Atmospheric Chemical ODE](https://mlsanity.com/paper/2408.01829/) (2024)
- [CosmoCLIP: Generalizing Large Vision-Language Models for Astronomical Imaging](https://mlsanity.com/paper/2407.07315/) (2024)
- [AstroMAE: Redshift Prediction Using a Masked Autoencoder with a Novel Fine-Tuning Architecture](https://mlsanity.com/paper/2409.01825/) (2024)
- [A review of unsupervised learning in astronomy](https://mlsanity.com/paper/2406.17316/) (2024)
- [Self-supervised Synthetic Pretraining for Inference of Stellar Mass Embedded in Dense Gas](https://mlsanity.com/paper/2510.24159/) (2025)
- [Semantic search for 100M+ galaxy images using AI-generated captions](https://mlsanity.com/paper/2512.11982/) (2025)
- [Galaxy Spin Classification I: Z-wise vs S-wise Spirals With Chirality Equivariant Residual Network](https://mlsanity.com/paper/2210.04168/) (2022)
- [Scalable Cosmic AI Inference using Cloud Serverless Computing](https://mlsanity.com/paper/2501.06249/) (2025)
- [Mantis Shrimp: Exploring Photometric Band Utilization in Computer Vision Networks for Photometric Redshift Estimation](https://mlsanity.com/paper/2501.09112/) (2025)
- [AstroCompress: A benchmark dataset for multi-purpose compression of astronomical data](https://mlsanity.com/paper/2506.08306/) (2025)
- [Can AI Dream of Unseen Galaxies? Conditional Diffusion Model for Galaxy Morphology Augmentation](https://mlsanity.com/paper/2506.16233/) (2025)

## Abstract (arXiv)

Sky surveys are the largest data generators in astronomy, making automated tools for extracting meaningful scientific information an absolute necessity. We show that, without the need for labels, self-supervised learning recovers representations of sky survey images that are semantically useful for a variety of scientific tasks. These representations can be directly used as features, or fine-tuned, to outperform supervised methods trained only on labeled data. We apply a contrastive learning framework on multi-band galaxy photometry from the Sloan Digital Sky Survey (SDSS) to learn image representations. We then use them for galaxy morphology classification, and fine-tune them for photometric redshift estimation, using labels from the Galaxy Zoo 2 dataset and SDSS spectroscopy. In both downstream tasks, using the same learned representations, we outperform the supervised state-of-the-art results, and we show that our approach can achieve the accuracy of supervised models while using 2-4 times fewer labels for training.