# Self-supervised Synthetic Pretraining for Inference of Stellar Mass Embedded in Dense Gas

arXiv: https://arxiv.org/abs/2510.24159 · HTML: https://mlsanity.com/paper/2510.24159/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 27 references that are in the map.
- First posted on arXiv 28 Oct 2025.

## Builds on (most cited first, 15 of 15)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [An Empirical Study of Training Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.02057/) (2021)
- [iBOT: Image BERT Pre-Training with Online Tokenizer](https://mlsanity.com/paper/2111.07832/) (2021)
- [Pre-training without Natural Images](https://mlsanity.com/paper/2101.08515/) (2021)
- [Learning to See by Looking at Noise](https://mlsanity.com/paper/2106.05963/) (2021)
- [Can Vision Transformers Learn without Natural Images?](https://mlsanity.com/paper/2103.13023/) (2021)
- [Procedural Image Programs for Representation Learning](https://mlsanity.com/paper/2211.16412/) (2022)
- [Pre-training Vision Transformers with Formula-driven Supervised Learning](https://mlsanity.com/paper/2206.09132/) (2022)
- [Self-Supervised Representation Learning for Astronomical Images](https://mlsanity.com/paper/2012.13083/) (2020)
- [Scaling Backwards: Minimal Synthetic Pre-training?](https://mlsanity.com/paper/2408.00677/) (2024)
- [Pre-training Vision Transformers with Very Limited Synthesized Images](https://mlsanity.com/paper/2307.14710/) (2023)
- [AstroM3: A self-supervised multimodal model for astronomy](https://mlsanity.com/paper/2411.08842/) (2024)

## Built on by (most cited first, 1 of 1)

- [Gestalt: a meta-foundation model for astronomy](https://mlsanity.com/paper/2609.38312/) (2026)

## Abstract (arXiv)

Stellar mass is a fundamental quantity that determines the properties and evolution of stars. However, estimating stellar masses in star-forming regions is challenging because young stars are obscured by dense gas and the regions are highly inhomogeneous, making spherical dynamical estimates unreliable. Supervised machine learning could link such complex structures to stellar mass, but it requires large, high-quality labeled datasets from high-resolution magneto-hydrodynamical (MHD) simulations, which are computationally expensive. We address this by pretraining a vision transformer on one million synthetic fractal images using the self-supervised framework DINOv2, and then applying the frozen model to limited high-resolution MHD simulations. Our results demonstrate that synthetic pretraining improves frozen-feature regression stellar mass predictions, with the pretrained model performing slightly better than a supervised model trained on the same limited simulations. Principal component analysis of the extracted features further reveals semantically meaningful structures, suggesting that the model enables unsupervised segmentation of star-forming regions without the need for labeled data or fine-tuning.