Transformers for scientific data: a pedagogical review for astronomers
Not on the map. Why?
Abstract (arXiv)
The deep learning architecture associated with ChatGPT and related generative AI products is known as transformers. Initially applied to Natural Language Processing, transformers and the self-attention mechanism they exploit have gained widespread interest across the natural sciences. The goal of this pedagogical and informal review is to introduce transformers to scientists. The review includes the mathematics underlying the attention mechanism, a description of the original transformer architecture, and a section on applications to time series and imaging data in astronomy. We include a Frequently Asked Questions section for readers who are curious about generative AI or interested in getting started with transformers for their research problem.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 8 of its 22 references that are in the map. First posted on arXiv 18 Oct 2023.
Builds on
8 of 22 references.
References · 8
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,375
- 2023Astroformer: More Data Might not be all you need for Classificationcited by 6
- 2022Deep Attention-Based Supernovae Classification of Multi-Band Light-Curvescited by 5
- 2022Don't Pay Attention to the Noise: Learning Self-supervised Representations of Light Curves with a Denoising Time Series Transformercited by 5
- 2023Distinguishing a planetary transit from false positives: a Transformer-based classification for planetary transit signalscited by 5
- 2022Astroconformer: Inferring Surface Gravity of Stars from Stellar Light Curves with Transformercited by 3
- 2022Detection of Strongly Lensed Arcs in Galaxy Clusters with Transformerscited by 2
Built on by
Citing papers per year: 2025: 1; 2026: 1.
2 papers.