# Improving astroBERT using Semantic Textual Similarity

arXiv: https://arxiv.org/abs/2212.00744 · HTML: https://mlsanity.com/paper/2212.00744/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 0 of its 3 references that are in the map.
- First posted on arXiv 29 Nov 2022.

## Built on by (most cited first, 3 of 3)

- [Meaning at the Planck scale? Contextualized word embeddings for doing history, philosophy, and sociology of science](https://mlsanity.com/paper/2411.14073/) (2024)
- [Astro-HEP-BERT: A bidirectional language model for studying the meanings of concepts in astrophysics and high energy physics](https://mlsanity.com/paper/2411.14877/) (2024)
- [Psychological Assessments with Large Language Models: A Privacy-Focused and Cost-Effective Approach](https://mlsanity.com/paper/2402.03435/) (2024)

## Abstract (arXiv)

The NASA Astrophysics Data System (ADS) is an essential tool for researchers that allows them to explore the astronomy and astrophysics scientific literature, but it has yet to exploit recent advances in natural language processing. At ADASS 2021, we introduced astroBERT, a machine learning language model tailored to the text used in astronomy papers in ADS. In this work we: - announce the first public release of the astroBERT language model; - show how astroBERT improves over existing public language models on astrophysics specific tasks; - and detail how ADS plans to harness the unique structure of scientific papers, the citation graph and citation context, to further improve astroBERT.