mlsanityData to 7 Oct 2026

OmniScience: A Domain-Specialized LLM for Scientific Reasoning and Discovery

arXiv 2503.17604 ↗First posted 22 Mar 2025cs.AIcited by 20

On the map: LLM reasoning → Knowledge- and tool-augmented reasoning

Abstract (arXiv)

Large Language Models (LLMs) have demonstrated remarkable potential in advancing scientific knowledge and addressing complex challenges. In this work, we introduce OmniScience, a specialized large reasoning model for general science, developed through three key components: (1) domain adaptive pretraining on a carefully curated corpus of scientific literature, (2) instruction tuning on a specialized dataset to guide the model in following domain-specific tasks, and (3) reasoning-based knowledge distillation through fine-tuning to significantly enhance its ability to generate contextually relevant and logically sound responses. We demonstrate the versatility of OmniScience by developing a battery agent that efficiently ranks molecules as potential electrolyte solvents or additives. Comprehensive evaluations reveal that OmniScience is competitive with state-of-the-art large reasoning models on the GPQA Diamond and domain-specific battery benchmarks, while outperforming all public reasoning and non-reasoning models with similar parameter counts. We further demonstrate via ablation experiments that domain adaptive pretraining and reasoning-based knowledge distillation are critical to attain our performance levels, across benchmarks.

Key facts. Cited by 20 papers in the map (data through 7 Oct 2026). Placed in Knowledge- and tool-augmented reasoning, one of 17 directions in LLM reasoning. Builds on 25 of its 49 references that are in the map. First posted on arXiv 22 Mar 2025.

Builds on

25 of 49 references.

References · 25

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  4. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  5. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  6. 2024Qwen2.5 Technical Reportcited by 4,713
  7. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  8. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
  9. 2023Qwen Technical Reportcited by 3,273
  10. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,829
  11. 2020Don't Stop Pretraining: Adapt Language Models to Domains and Taskscited by 1,548
  12. 2025s1: Simple test-time scalingcited by 1,311
13 more
  1. 2021Improving language models by retrieving from trillions of tokenscited by 1,278
  2. 2022Atlas: Few-shot Learning with Retrieval Augmented Language Modelscited by 940
  3. 2023BloombergGPT: A Large Language Model for Financecited by 923
  4. 2023Gemini: A Family of Highly Capable Multimodal Modelscited by 810
  5. 2022Galactica: A Large Language Model for Sciencecited by 736
  6. 2020ChemBERTa: Large-Scale Self-Supervised Pretraining for Molecular Property Predictioncited by 332
  7. 2019NeMo: a toolkit for building AI applications using Neural Modulescited by 230
  8. 2023ChipNeMo: Domain-Adapted LLMs for Chip Designcited by 168
  9. 2024BioMedLM: A 2.7B Parameter Language Model Trained On Biomedical Textcited by 85
  10. 2024Tag-LLM: Repurposing General-Purpose LLMs for Specialized Domainscited by 30
  11. 2025MatterChat: A Multi-Modal LLM for Material Sciencecited by 18
  12. 2024Adapting Amidst Degradation: Cross Domain Li-ion Battery Health Estimation via Physics-Guided Test-Time Trainingcited by 3
  13. 2025Battery State of Health Estimation Using LLM Frameworkcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 14; 2026: 6.

20 papers.

Sort:

By area:

  1. 2025From AI for Science to Agentic Science: A Survey on Autonomous Scientific Discoverycited by 74
  2. 2025Paper2Code: Automating Code Generation from Scientific Papers in Machine Learningcited by 67
  3. 2025Training a Scientific Reasoning Model for Chemistrycited by 54
  4. 2025History Rhymes: Accelerating LLM Reinforcement Learning with RhymeRLcited by 35
  5. 2025A Survey of Scientific Large Language Models: From Data Foundations to Agent Frontierscited by 34
  6. 2025Front-Loading Reasoning: The Synergy between Pretraining and Post-Training Datacited by 26
  7. 2025PiFlow: Principle-Aware Scientific Discovery with Multi-Agent Collaborationcited by 25
  8. 2025RLBoost: Harvesting Preemptible Resources for Cost-Efficient Reinforcement Learning on LLMscited by 10
  9. 2025Toward Scientific Reasoning in LLMs: Training from Expert Discussions via Reinforcement Learningcited by 6
  10. 2025Demystifying Scientific Problem-Solving in LLMs by Probing Knowledge and Reasoningcited by 4
  11. 2026WildSci: Advancing Scientific Reasoning from In-the-Wild Literaturecited by 4
  12. 2026JUDO: A Juxtaposed Domain-Oriented Multimodal Reasoner for Industrial Anomaly QAcited by 4
8 more
  1. 2026MATRIX: A Multimodal Benchmark and Post-Training Framework for Materials Sciencecited by 3
  2. 2026Evaluating the Progression of Large Language Model Capabilities for Small-Molecule Drug Designcited by 3
  3. 2025Make Still Further Progress: Chain of Thoughts for Tabular Data Leaderboardcited by 2
  4. 2025RAISE: Enhancing Scientific Reasoning in LLMs via Step-by-Step Retrievalcited by 1
  5. 2025Kunlun Anomaly Troubleshooter: Enabling Kernel-Level Anomaly Detection and Causal Reasoning for Large Model Distributed Inferencecited by 1
  6. 2025Training-Free Active Learning Framework in Materials Science with Large Language Modelscited by 1
  7. 2026A unified multimodal understanding and generation model for cross-disciplinary scientific researchcited by 1
  8. 2026Self-Refining Topology Optimization via an LLM-Based Multi-Agent Frameworkcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026FALSIFYBENCH: Evaluating Inductive Reasoning in LLMs with Rule Discovery Games
  2. 2026S1-Omni: A Unified Multimodal Reasoning Model for Scientific Understanding, Prediction, and Generation
  3. 2025MegaScience: Pushing the Frontiers of Post-Training Datasets for Science Reasoning
  4. 2025CURIE: Evaluating LLMs On Multitask Scientific Long Context Understanding and Reasoning
  5. 2025SciReasoner: Laying the Scientific Reasoning Ground Across Disciplines
  6. 2024SciKnowEval: Evaluating Multi-level Scientific Knowledge of Large Language Models
  7. 2025ChemATP: A Training-Free Chemical Reasoning Framework for Large Language Models
  8. 2025Towards Scientific Intelligence: A Survey of LLM-based Scientific Agents