mlsanityData to 14 Oct 2026

DARWIN Series: Domain Specific Large Language Models for Natural Science

arXiv 2308.13565 ↗First posted 25 Aug 2023cs.CL cond-mat.mtrl-sci physics.app-phcited by 39

Not on the map. Why?

Abstract (arXiv)

Emerging tools bring forth fresh approaches to work, and the field of natural science is no different. In natural science, traditional manual, serial, and labour-intensive work is being augmented by automated, parallel, and iterative processes driven by artificial intelligence-based experimental automation and more. To add new capabilities in natural science, enabling the acceleration and enrichment of automation of the discovery process, we present DARWIN, a series of tailored LLMs for natural science, mainly in physics, chemistry, and material science. This series relies on open-source LLM, incorporating structured and unstructured scientific knowledge from public datasets and literature. We fine-tuned the models using over 60,000 instruction data points, emphasizing factual correctness. During the fine-tuning, we introduce the Scientific Instruction Generation (SIG) model, automating instruction generation from scientific texts. This eliminates the need for manual extraction or domain-specific knowledge graphs and efficiently injects scientific knowledge into the model. We also explore multi-task training strategies, revealing interconnections between scientific tasks. DARWIN series not only achieves state-of-the-art results on various scientific tasks but also diminishes reliance on closed-source AI models. Our research showcases the ability of LLM in the scientific domain, with the overarching goal of fostering prosperity within the broader AI for science community.

Key facts. Cited by 39 papers in the map (data through 14 Oct 2026). Builds on 19 of its 49 references that are in the map. First posted on arXiv 25 Aug 2023.

Builds on

19 of 49 references.

References · 19

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2022Training language models to follow instructions with human feedbackcited by 15,971
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  5. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,474
  6. 2022Scaling Instruction-Finetuned Language Modelscited by 3,024
  7. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,724
  8. 2022Large Language Models Encode Clinical Knowledgecited by 1,830
  9. 2019SciBERT: A Pretrained Language Model for Scientific Textcited by 1,414
  10. 2020Domain-Specific Language Model Pretraining for Biomedical Natural Language Processingcited by 1,009
  11. 2022Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Taskscited by 868
  12. 2023RWKV: Reinventing RNNs for the Transformer Eracited by 827
7 more
  1. 2017Crowdsourcing Multiple Choice Science Questionscited by 765
  2. 2022Galactica: A Large Language Model for Sciencecited by 744
  3. 2022BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Miningcited by 538
  4. 2023The Curse of Recursion: Training on Generated Data Makes Models Forgetcited by 356
  5. 2023HuaTuo: Tuning LLaMA Model with Chinese Medical Knowledgecited by 162
  6. 2023Mol-Instructions: A Large-Scale Biomolecular Instruction Dataset for Large Language Modelscited by 133
  7. 2023Large Language Models as Master Key: Unlocking the Secrets of Materials Science with GPTcited by 30

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 2; 2024: 21; 2025: 13; 2026: 3.

39 papers.

Sort:

By area:

  1. 2024Continual Learning of Large Language Models: A Comprehensive Surveycited by 234
  2. 2024A Review of Large Language Models and Autonomous Agents in Chemistrycited by 99
  3. 2024Scientific Large Language Models: A Survey on Biological & Chemical Domainscited by 80
  4. 2025From AI for Science to Agentic Science: A Survey on Autonomous Scientific Discoverycited by 77
  5. 2024Survey on Knowledge Distillation for Large Language Models: Methods, Evaluation, and Applicationcited by 56
  6. 2024Developing ChemDFM as a large language foundation model for chemistrycited by 52
  7. 2024SciRIFF: A Resource to Enhance Language Model Instruction-Following over Scientific Literaturecited by 41
  8. 2024Are LLMs Ready for Real-World Materials Discovery?cited by 40
  9. 2025A Survey of Scientific Large Language Models: From Data Foundations to Agent Frontierscited by 37
  10. 2023GeoGalactica: A Scientific Large Language Model in Geosciencecited by 33
  11. 2025Position: Multimodal Large Language Models Can Significantly Advance Scientific Reasoningcited by 33
  12. 2024LLaMP: Large Language Model Made Powerful for High-fidelity Materials Knowledge Retrieval and Distillationcited by 32
27 more
  1. 2025PiFlow: Principle-Aware Scientific Discovery with Multi-Agent Collaborationcited by 28
  2. 2024PLLaMa: An Open-source Large Language Model for Plant Sciencecited by 22
  3. 2024LLM4Mat-Bench: Benchmarking Large Language Models for Materials Property Predictioncited by 21
  4. 2025ChemDFM-R: A Chemical Reasoning LLM Enhanced with Atomized Chemical Knowledgecited by 20
  5. 2024MatExpert: Decomposing Materials Discovery by Mimicking Human Expertscited by 18
  6. 2024Leveraging Biomolecule and Natural Language through Multi-Modal Learning: A Surveycited by 17
  7. 2024Construction and Application of Materials Knowledge Graph in Multidisciplinary Materials Science via Large Language Modelcited by 15
  8. 2025Large language models as uncertainty-calibrated optimizers for experimental discoverycited by 13
  9. 2025TeachLM: Post-Training LLMs for Education Using Authentic Learning Datacited by 9
  10. 2025PriM: Principle-Inspired Material Discovery through Multi-Agent Collaborationcited by 7
  11. 2025Toward Scientific Reasoning in LLMs: Training from Expert Discussions via Reinforcement Learningcited by 7
  12. 2025Research on Personalized Financial Product Recommendation by Integrating Large Language Models and Graph Neural Networkscited by 6
  13. 2025CoRank: LLM-Based Compact Reranking with Document Features for Scientific Retrievalcited by 5
  14. 2025MiST: Understanding the Role of Mid-Stage Scientific Training in Developing Chemical Reasoning Modelscited by 3
  15. 2024Enhancing Exchange Rate Forecasting with Explainable Deep Learning Modelscited by 2
  16. 2023Speculative Exploration on the Concept of Artificial Agents Conducting Autonomous Researchcited by 1
  17. 2024DGoT: Dynamic Graph of Thoughts for Scientific Abstract Generationcited by 1
  18. 2024Dynamic In-context Learning with Conversational Models for Data Extraction and Materials Property Predictioncited by 1
  19. 2024Enhancing Spectral Knowledge Interrogation: A Reliable Retrieval-Augmented Generative Framework on Large Language Modelscited by 1
  20. 2024ByteScience: Bridging Unstructured Scientific Literature and Structured Data with Auto Fine-tuned Large Language Model in Token Granularitycited by 1
  21. 2026A unified multimodal understanding and generation model for cross-disciplinary scientific researchcited by 1
  22. 2024Dual-Label Learning With Irregularly Present Labelscited by 0
  23. 2024Polymetis:Large Language Modeling for Multiple Material Domainscited by 0
  24. 2024Scaling Particle Collision Data Analysiscited by 0
  25. 2025Role of Large Language Models and Retrieval-Augmented Generation for Accelerating Crystalline Material Discovery: A Systematic Reviewcited by 0
  26. 2026Using a Human-AI Teaming Approach to Create and Curate Scientific Datasets with the SCILIRE Systemcited by 0
  27. 2026Team MKC at CLPsych 2026: Capturing and Characterizing Mental Health Changes through Social Media Timeline Dynamicscited by 0