mlsanityData to 7 Oct 2026

A brief review of contrastive learning applied to astrophysics

arXiv 2306.05528 ↗First posted 8 Jun 2023astro-ph.IM cs.LGcited by 5

Not on the map. Why?

Abstract (arXiv)

Reliable tools to extract patterns from high-dimensionality spaces are becoming more necessary as astronomical datasets increase both in volume and complexity. Contrastive Learning is a self-supervised machine learning algorithm that extracts informative measurements from multi-dimensional datasets, which has become increasingly popular in the computer vision and Machine Learning communities in recent years. To do so, it maximizes the agreement between the information extracted from augmented versions of the same input data, making the final representation invariant to the applied transformations. Contrastive Learning is particularly useful in astronomy for removing known instrumental effects and for performing supervised classifications and regressions with a limited amount of available labels, showing a promising avenue towards Foundation Models. This short review paper briefly summarizes the main concepts behind contrastive learning and reviews the first promising applications to astronomy. We include some practical recommendations on which applications are particularly attractive for contrastive learning.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 43 of its 88 references that are in the map. First posted on arXiv 8 Jun 2023.

Builds on

43 of 88 references.

References · 43

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  4. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  5. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  6. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  7. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  8. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  9. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  10. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  11. 2018Neural Ordinary Differential Equationscited by 3,733
  12. 2016Improved Techniques for Training GANscited by 2,938
31 more
  1. 2020Exploring Simple Siamese Representation Learningcited by 2,853
  2. 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,841
  3. 2020Supervised Contrastive Learningcited by 2,803
  4. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  5. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,282
  6. 2015Variational Inference with Normalizing Flowscited by 2,161
  7. 2015Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networkscited by 2,048
  8. 2016Density estimation using Real NVPcited by 2,029
  9. 2019Contrastive Multiview Codingcited by 1,297
  10. 2018Learning deep representations by mutual information estimation and maximizationcited by 1,265
  11. 2019Self-Supervised Learning of Pretext-Invariant Representationscited by 806
  12. 2017Masked Autoregressive Flow for Density Estimationcited by 680
  13. 2016Improving Variational Inference with Inverse Autoregressive Flowcited by 636
  14. 2017Wasserstein GANcited by 635
  15. 2018FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Modelscited by 584
  16. 2016Tutorial on Variational Autoencoderscited by 404
  17. 2019Your Classifier is Secretly an Energy Based Model and You Should Treat it Like Onecited by 402
  18. 2020Keystroke Dynamics as Part of Lifeloggingcited by 334
  19. 2020Universal Domain Adaptation through Self Supervisioncited by 172
  20. 2021Cross-Domain Adaptive Clustering for Semi-Supervised Domain Adaptationcited by 67
  21. 2020Self-Supervised Representation Learning for Astronomical Imagescited by 18
  22. 2022Astronomia ex machina: a history, primer, and outlook on neural networks in astronomycited by 17
  23. 2019Statistically-informed deep learning for gravitational wave parameter estimationcited by 14
  24. 2021Practical Galaxy Morphology Tools from Deep Supervised Representation Learningcited by 14
  25. 2021Self-supervised similarity search for large scientific datasetscited by 11
  26. 2022Towards Galaxy Foundation Models with Hybrid Contrastive Learningcited by 10
  27. 2023DeepAstroUDA: Semi-Supervised Universal Domain Adaptation for Cross-Survey Galaxy Morphology Classification and Anomaly Detectioncited by 8
  28. 2020DeepMerge: Classifying High-redshift Merging Galaxies with Deep Neural Networkscited by 5
  29. 2021Estimating Galactic Distances From Images Using Self-supervised Representation Learningcited by 4
  30. 2022A Machine Learning Enhanced Approach for Automated Sunquake Detection in Acoustic Emission Mapscited by 4
  31. 2021A Survey on Deep Domain Adaptation and Tiny Object Detection Challenges, Techniques and Datasetscited by 3

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 3; 2025: 1.

5 papers.

  1. 2024PAPERCLIP: Associating Astronomical Observations and Natural Language with Multi-Modal Modelscited by 15
  2. 2024AstroPT: Scaling Large Observation Models for Astronomycited by 9
  3. 2023Data Compression and Inference in Cosmology with Self-Supervised Machine Learningcited by 5
  4. 2024A review of unsupervised learning in astronomycited by 2
  5. 2025Detecting Model Misspecification in Cosmology with Scale-Dependent Normalizing Flowscited by 2