# ASTROMER: A transformer-based embedding for the representation of light curves

arXiv: https://arxiv.org/abs/2205.01677 · HTML: https://mlsanity.com/paper/2205.01677/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 0 of its 13 references that are in the map.
- First posted on arXiv 2 May 2022.

## Built on by (most cited first, 9 of 9)

- [Astronomia ex machina: a history, primer, and outlook on neural networks in astronomy](https://mlsanity.com/paper/2211.03796/) (2022)
- [AstroMAE: Redshift Prediction Using a Masked Autoencoder with a Novel Fine-Tuning Architecture](https://mlsanity.com/paper/2409.01825/) (2024)
- [The Scaling Law in Stellar Light Curves](https://mlsanity.com/paper/2405.17156/) (2024)
- [Self-Supervised Learning for Modeling Gamma-ray Variability in Blazars](https://mlsanity.com/paper/2302.07700/) (2023)
- [Identifying Light-curve Signals with a Deep Learning Based Object Detection Algorithm. II. A General Light Curve Classification Framework](https://mlsanity.com/paper/2311.08080/) (2023)
- [Image-Based Multi-Survey Classification of Light Curves with a Pre-Trained Vision Transformer](https://mlsanity.com/paper/2507.11711/) (2025)
- [ASTROCO: Self-Supervised Conformer-Style Transformers for Light-Curve Embeddings](https://mlsanity.com/paper/2509.24134/) (2025)
- [Simulation-Based Pretraining and Domain Adaptation for Astronomical Time Series with Minimal Labeled Data](https://mlsanity.com/paper/2510.12958/) (2025)
- [Transformer Learning of Chaotic Collective Dynamics in Many-Body Systems](https://mlsanity.com/paper/2601.19080/) (2026)

## Abstract (arXiv)

Taking inspiration from natural language embeddings, we present ASTROMER, a transformer-based model to create representations of light curves. ASTROMER was pre-trained in a self-supervised manner, requiring no human-labeled data. We used millions of R-band light sequences to adjust the ASTROMER weights. The learned representation can be easily adapted to other surveys by re-training ASTROMER on new sources. The power of ASTROMER consists of using the representation to extract light curve embeddings that can enhance the training of other models, such as classifiers or regressors. As an example, we used ASTROMER embeddings to train two neural-based classifiers that use labeled variable stars from MACHO, OGLE-III, and ATLAS. In all experiments, ASTROMER-based classifiers outperformed a baseline recurrent neural network trained on light curves directly when limited labeled data was available. Furthermore, using ASTROMER embeddings decreases computational resources needed while achieving state-of-the-art results. Finally, we provide a Python library that includes all the functionalities employed in this work. The library, main code, and pre-trained weights are available at https://github.com/astromer-science