mlsanityData to 7 Oct 2026

AstroLLaMA: Towards Specialized Foundation Models in Astronomy

arXiv 2309.06126 ↗First posted 12 Sep 2023astro-ph.IM astro-ph.CO astro-ph.GAcited by 41

Not on the map. Why?

Abstract (arXiv)

Large language models excel in many human-language tasks but often falter in highly specialized domains like scholarly astronomy. To bridge this gap, we introduce AstroLLaMA, a 7-billion-parameter model fine-tuned from LLaMA-2 using over 300,000 astronomy abstracts from arXiv. Optimized for traditional causal language modeling, AstroLLaMA achieves a 30% lower perplexity than Llama-2, showing marked domain adaptation. Our model generates more insightful and scientifically relevant text completions and embedding extraction than state-of-the-arts foundation models despite having significantly fewer parameters. AstroLLaMA serves as a robust, domain-specific model with broad fine-tuning potential. Its public release aims to spur astronomy-focused research, including automatic paper summarization and conversational agent development.

Key facts. Cited by 41 papers in the map (data through 7 Oct 2026). Builds on 12 of its 18 references that are in the map. First posted on arXiv 12 Sep 2023.

Builds on

12 of 18 references.

References · 12

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2023Visual Instruction Tuningcited by 7,792
  5. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  6. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
  7. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,973
  8. 2021Building astroBERT, a language model for Astronomy & Astrophysicscited by 18
  9. 2023Harnessing the Power of Adversarial Prompting and Large Language Models for Robust Hypothesis Generation in Astronomycited by 11
  10. 2015ADS: The Next Generation Search Platformcited by 5
  11. 2023Galactic ChitChat: Using Large Language Models to Converse with Astronomy Literaturecited by 4
  12. 2021From Data Processes to Data Products: Knowledge Infrastructures in Astronomycited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 13; 2025: 18; 2026: 9.

41 papers.

Sort:

By area:

  1. 2024A Survey on Knowledge Distillation of Large Language Modelscited by 276
  2. 2024Continual Learning of Large Language Models: A Comprehensive Surveycited by 228
  3. 2024A Comprehensive Survey of Scientific Large Language Models and Their Applications in Scientific Discoverycited by 96
  4. 2025Towards Scientific Intelligence: A Survey of LLM-based Scientific Agentscited by 96
  5. 2023Multiple Physics Pretraining for Physical Surrogate Modelscited by 90
  6. 2025From AI for Science to Agentic Science: A Survey on Autonomous Scientific Discoverycited by 74
  7. 2025A Survey of Scientific Large Language Models: From Data Foundations to Agent Frontierscited by 34
  8. 2024PLLaMa: An Open-source Large Language Model for Plant Sciencecited by 22
  9. 2025Large Physics Models: Towards a collaborative approach with Large Language Models and Foundation Modelscited by 14
  10. 2024AstroLLaMA-Chat: Scaling AstroLLaMA with Conversational and Diverse Datasetscited by 13
  11. 2024Geospatial foundation models for image analysis: evaluating and enhancing NASA-IBM Prithvi's domain adaptabilitycited by 11
  12. 2024AstroPT: Scaling Large Observation Models for Astronomycited by 9
29 more
  1. 2024Delving into the Utilisation of ChatGPT in Scientific Publications in Astronomycited by 7
  2. 2024Coarse-to-Fine Highlighting: Reducing Knowledge Hallucination in Large Language Modelscited by 7
  3. 2025LoRA-Leak: Membership Inference Attacks Against LoRA Fine-tuned Language Modelscited by 7
  4. 2024AstroMLab 2: AstroLLaMA-2-70B Model and Benchmarking Specialised LLMs for Astronomycited by 6
  5. 2025AstroLLaVA: towards the unification of astronomical data and natural languagecited by 6
  6. 2025Towards a Physics Foundation Modelcited by 5
  7. 2025The Platonic Universe: Do Foundation Models See the Same Sky?cited by 5
  8. 2025Agentic Exploration of Physics Modelscited by 5
  9. 2025A Short Survey on Small Reasoning Models: Training, Inference, Applications and Research Directionscited by 4
  10. 2024Designing an Evaluation Framework for Large Language Models in Astronomy Researchcited by 3
  11. 2024Question Rephrasing for Quantifying Uncertainty in Large Language Models: Applications in Molecular Chemistry Taskscited by 3
  12. 2025Physics Steering: Causal Control of Cross-Domain Concepts in a Physics Foundation Modelcited by 3
  13. 2025A Survey of Large Language Models in Discipline-specific Research: Challenges, Methods and Opportunitiescited by 2
  14. 2025From Queries to Criteria: Understanding How Astronomers Evaluate LLMscited by 2
  15. 2025FeynTune: Large Language Models for High-Energy Theorycited by 2
  16. 2024ORBIT: Cost-Effective Dataset Curation for Large Language Model Domain Adaptation with an Astronomy Case Studycited by 1
  17. 2025From Raw Corpora to Domain Benchmarks: Automated Evaluation of LLM Domain Expertisecited by 1
  18. 2026Discovery of Interpretable Surrogates via Agentic AI: Application to Gravitational Wavescited by 1
  19. 2025Smotrom tvoja pa ander drogoj verden! Resurrecting Dead Pidgin with Generative Models: Russenorsk Case Studycited by 0
  20. 2025Bridging the Digital Divide: Small Language Models as a Pathway for Physics and Photonics Education in Underdeveloped Regionscited by 0
  21. 2025AstroReview: An LLM-driven Multi-Agent Framework for Telescope Proposal Peer Review and Refinementcited by 0
  22. 2026Performance of Small Language Model Pretraining on FABRIC: An Empirical Studycited by 0
  23. 2026EVE: A Domain-Specific LLM Framework for Earth Intelligencecited by 0
  24. 2026AstroRAG -- A Pagerank-Based Retrieval-Augmented Generation Pipeline for Question Answering in Astronomycited by 0
  25. 2026SciLENS: RL-Driven Autonomous Agents for Scientific Localized Evidence Navigation and Synthesiscited by 0
  26. 2026AstroSpecLM: A Spectrum-Language Model for Evidence-Grounded Astronomical Spectral Analysiscited by 0
  27. 2026AquiLLM: Evaluating Faithfulness in Open-Weight RAG-LLM Systems for Scientific Researchcited by 0
  28. 2026Rethinking Domain Specialization for Open-Ended Scientific Reasoning in Astronomy Language Modelscited by 0
  29. 2026Gestalt: a meta-foundation model for astronomycited by 0