# Simulation-Based Pretraining and Domain Adaptation for Astronomical Time Series with Minimal Labeled Data

arXiv: https://arxiv.org/abs/2510.12958 · HTML: https://mlsanity.com/paper/2510.12958/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 15 of its 40 references that are in the map.
- First posted on arXiv 14 Oct 2025.

## Builds on (most cited first, 15 of 15)

- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [Supervised Contrastive Learning](https://mlsanity.com/paper/2004.11362/) (2020)
- [Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling](https://mlsanity.com/paper/1412.3555/) (2014)
- [RAPID: Early Classification of Explosive Transients using Deep Learning](https://mlsanity.com/paper/1904.00014/) (2019)
- [Practical Galaxy Morphology Tools from Deep Supervised Representation Learning](https://mlsanity.com/paper/2110.12735/) (2021)
- [A Deep Learning Approach for Active Anomaly Detection of Extragalactic Transients](https://mlsanity.com/paper/2103.12102/) (2021)
- [ASTROMER: A transformer-based embedding for the representation of light curves](https://mlsanity.com/paper/2205.01677/) (2022)
- [Real-Time Detection of Anomalies in Large-Scale Transient Surveys](https://mlsanity.com/paper/2111.00036/) (2021)
- [A Classifier-Based Approach to Multi-Class Anomaly Detection for Astronomical Transients](https://mlsanity.com/paper/2403.14742/) (2024)
- [Photo-zSNthesis: Converting Type Ia Supernova Lightcurves to Redshift Estimates via Deep Learning](https://mlsanity.com/paper/2305.11869/) (2023)
- [Transfer Learning for Transient Classification: From Simulations to Real Data and ZTF to LSST](https://mlsanity.com/paper/2502.18558/) (2025)
- [Causal Foundation Models: Disentangling Physics from Instrument Properties](https://mlsanity.com/paper/2507.05333/) (2025)
- [Predicting the Age of Astronomical Transients from Real-Time Multivariate Time Series](https://mlsanity.com/paper/2311.17143/) (2023)

## Abstract (arXiv)

Astronomical time-series analysis faces a critical limitation: the scarcity of labeled observational data. We present a pre-training approach that leverages simulations, significantly reducing the need for labeled examples from real observations. Our models, trained on simulated data from multiple astronomical surveys (ZTF and LSST), learn generalizable representations that transfer effectively to downstream tasks. Using classifier-based architectures enhanced with contrastive and adversarial objectives, we create domain-agnostic models that demonstrate substantial performance improvements over baseline methods in classification, redshift estimation, and anomaly detection when fine-tuned with minimal real data. Remarkably, our models exhibit effective zero-shot transfer capabilities, achieving comparable performance on future telescope (LSST) simulations when trained solely on existing telescope (ZTF) data. Furthermore, they generalize to very different astronomical phenomena (namely variable stars from NASA's textitKepler telescope) despite being trained on transient events, demonstrating cross-domain capabilities. Our approach provides a practical solution for building general models when labeled data is scarce, but domain knowledge can be encoded in simulations.