Probing Scientific General Intelligence of LLMs with Scientist-Aligned Workflows
Not on the map. Why?
Abstract (arXiv)
Despite advances in scientific AI, a coherent framework for Scientific General Intelligence (SGI)-the ability to autonomously conceive, investigate, and reason across scientific domains-remains lacking. We present an operational SGI definition grounded in the Practical Inquiry Model (PIM: Deliberation, Conception, Action, Perception) and operationalize it via four scientist-aligned tasks: deep research, idea generation, dry/wet experiments, and experimental reasoning. SGI-Bench comprises over 1,000 expert-curated, cross-disciplinary samples inspired by Science's 125 Big Questions, enabling systematic evaluation of state-of-the-art LLMs. Results reveal gaps: low exact match (10--20%) in deep research despite step-level alignment; ideas lacking feasibility and detail; high code executability but low execution result accuracy in dry experiments; low sequence fidelity in wet protocols; and persistent multimodal comparative-reasoning challenges. We further introduce Test-Time Reinforcement Learning (TTRL), which optimizes retrieval-augmented novelty rewards at inference, enhancing hypothesis novelty without reference answer. Together, our PIM-grounded definition, workflow-centric benchmark, and empirical insights establish a foundation for AI systems that genuinely participate in scientific discovery.
Key facts. Cited by 25 papers in the map (data through 7 Oct 2026). First posted on arXiv 18 Dec 2025.
Builds on
0 of 0 references.
No references here yet.
Built on by
Citing papers per year: 2025: 1; 2026: 24.
25 papers.
Sort:
By area:
- 2026InternAgent-1.5: A Unified Agentic Framework for Long-Horizon Autonomous Scientific Discoverycited by 24
- 2026InternVL-U: Democratizing Unified Multimodal Models for Understanding, Reasoning, Generation and Editingcited by 23
- 2026CreativeBench: Benchmarking and Enhancing Machine Creativity via Self-Evolving Challengescited by 9
- 2026DeepInnovator: Triggering the Innovative Capabilities of LLMscited by 4
- 2026SciIntegrity-Bench: A Benchmark for Evaluating Academic Integrity in AI Scientist Systemscited by 4
- 2025SciEvalKit: An Open-source Evaluation Toolkit for Scientific General Intelligencecited by 3
- 2026Evaluating the Progression of Large Language Model Capabilities for Small-Molecule Drug Designcited by 3
- 2026Intern-S2-Preview: Scientific Agentic Foundation Modelcited by 3
- 2026SciDataCopilot: An Agentic Data Preparation Framework for AGI-driven Scientific Discoverycited by 2
- 2026HarmonyCell: Automating Single-Cell Perturbation Modeling under Semantic and Distribution Shiftscited by 2
- 2026FlowPIE: Test-Time Scientific Idea Evolution with Flow-Guided Literature Explorationcited by 2
- 2026StraTA: Incentivizing Agentic Reinforcement Learning with Strategic Trajectory Abstractioncited by 2
13 more
- 2026AgentCompass: A Unified Evaluation Infrastructure for Agent Capabilitiescited by 2
- 2026Beyond Gemini-3-Pro: Revisiting LLM Routing and Aggregation at Scalecited by 1
- 2026Learning Domain Knowledge in Multimodal Large Language Models through Reinforcement Fine-Tuningcited by 1
- 2026Embodied Science: Closing the Discovery Loop with Agentic Embodied AIcited by 1
- 2026BilliardPhys-Bench: Benchmarking Physical Reasoning and Visual Dynamics of Multimodal LLMscited by 1
- 2026SCI-PRM: A Tool Aware Process Reward Model for Scientific Reasoning Verificationcited by 1
- 2026TruthInsightBench: An Evidence-Grounded Benchmark for Automated Evaluation of Open-Ended Scientific Discovery Agentscited by 1
- 2026AblateCell: A Reproduce-then-Ablate Agent for Virtual Cell Repositoriescited by 0
- 2026ReCrit: Transition-Aware Reinforcement Learning for Scientific Critic Reasoningcited by 0
- 2026SciOrch: Learning to Orchestrate Expert LLMs for Solving Frontier Multimodal Scientific Reasoning Taskscited by 0
- 2026Ideas Have Genomes: Benchmarking Scientific Lineage Reasoning and Lineage-Grounded Idea Generationcited by 0
- 2026Tracing the Cascade: A Topology-Aware Evaluation Framework for Scientific Agent Hallucinationscited by 0
- 2026MindFlow: Mind Supernet Powered Thinking Flows for Research Idea Innovationcited by 0
- 2026MindFlow: Mind Supernet Powered Thinking Flows for Research Idea Innovation8 Oct 2026
- 2026TruthInsightBench: An Evidence-Grounded Benchmark for Automated Evaluation of Open-Ended Scientific Discovery Agents4 Sep 2026
- 2026Intern-S2-Preview: Scientific Agentic Foundation Model13 Aug 2026
- 2026Tracing the Cascade: A Topology-Aware Evaluation Framework for Scientific Agent Hallucinations1 Aug 2026
- 2026AgentCompass: A Unified Evaluation Infrastructure for Agent Capabilities15 Jul 2026
- 2026Ideas Have Genomes: Benchmarking Scientific Lineage Reasoning and Lineage-Grounded Idea Generation9 Jul 2026
- 2026SciOrch: Learning to Orchestrate Expert LLMs for Solving Frontier Multimodal Scientific Reasoning Tasks14 Jun 2026
- 2026SCI-PRM: A Tool Aware Process Reward Model for Scientific Reasoning Verification3 Jun 2026
- 2026BilliardPhys-Bench: Benchmarking Physical Reasoning and Visual Dynamics of Multimodal LLMs29 May 2026
- 2026ReCrit: Transition-Aware Reinforcement Learning for Scientific Critic Reasoning11 May 2026
- 2026SciIntegrity-Bench: A Benchmark for Evaluating Academic Integrity in AI Scientist Systems11 May 2026
- 2026StraTA: Incentivizing Agentic Reinforcement Learning with Strategic Trajectory Abstraction7 May 2026
13 more
- 2026AblateCell: A Reproduce-then-Ablate Agent for Virtual Cell Repositories21 Apr 2026
- 2026Evaluating the Progression of Large Language Model Capabilities for Small-Molecule Drug Design17 Apr 2026
- 2026FlowPIE: Test-Time Scientific Idea Evolution with Flow-Guided Literature Exploration31 Mar 2026
- 2026Embodied Science: Closing the Discovery Loop with Agentic Embodied AI20 Mar 2026
- 2026CreativeBench: Benchmarking and Enhancing Machine Creativity via Self-Evolving Challenges12 Mar 2026
- 2026InternVL-U: Democratizing Unified Multimodal Models for Understanding, Reasoning, Generation and Editing10 Mar 2026
- 2026HarmonyCell: Automating Single-Cell Perturbation Modeling under Semantic and Distribution Shifts2 Mar 2026
- 2026DeepInnovator: Triggering the Innovative Capabilities of LLMs21 Feb 2026
- 2026SciDataCopilot: An Agentic Data Preparation Framework for AGI-driven Scientific Discovery9 Feb 2026
- 2026InternAgent-1.5: A Unified Agentic Framework for Long-Horizon Autonomous Scientific Discovery9 Feb 2026
- 2026Learning Domain Knowledge in Multimodal Large Language Models through Reinforcement Fine-Tuning23 Jan 2026
- 2026Beyond Gemini-3-Pro: Revisiting LLM Routing and Aggregation at Scale4 Jan 2026
- 2025SciEvalKit: An Open-source Evaluation Toolkit for Scientific General Intelligence26 Dec 2025
- 2026InternAgent-1.5: A Unified Agentic Framework for Long-Horizon Autonomous Scientific Discoverycited by 24
- 2026DeepInnovator: Triggering the Innovative Capabilities of LLMscited by 4
- 2026Intern-S2-Preview: Scientific Agentic Foundation Modelcited by 3
- 2026SciDataCopilot: An Agentic Data Preparation Framework for AGI-driven Scientific Discoverycited by 2
- 2026StraTA: Incentivizing Agentic Reinforcement Learning with Strategic Trajectory Abstractioncited by 2
- 2026AgentCompass: A Unified Evaluation Infrastructure for Agent Capabilitiescited by 2
- 2026Embodied Science: Closing the Discovery Loop with Agentic Embodied AIcited by 1
- 2026TruthInsightBench: An Evidence-Grounded Benchmark for Automated Evaluation of Open-Ended Scientific Discovery Agentscited by 1
- 2026Tracing the Cascade: A Topology-Aware Evaluation Framework for Scientific Agent Hallucinationscited by 0
- 2026TruthInsightBench: An Evidence-Grounded Benchmark for Automated Evaluation of Open-Ended Scientific Discovery Agents4 Sep 2026
- 2026Intern-S2-Preview: Scientific Agentic Foundation Model13 Aug 2026
- 2026Tracing the Cascade: A Topology-Aware Evaluation Framework for Scientific Agent Hallucinations1 Aug 2026
- 2026AgentCompass: A Unified Evaluation Infrastructure for Agent Capabilities15 Jul 2026
- 2026StraTA: Incentivizing Agentic Reinforcement Learning with Strategic Trajectory Abstraction7 May 2026
- 2026Embodied Science: Closing the Discovery Loop with Agentic Embodied AI20 Mar 2026
- 2026DeepInnovator: Triggering the Innovative Capabilities of LLMs21 Feb 2026
- 2026SciDataCopilot: An Agentic Data Preparation Framework for AGI-driven Scientific Discovery9 Feb 2026
- 2026InternAgent-1.5: A Unified Agentic Framework for Long-Horizon Autonomous Scientific Discovery9 Feb 2026
- 2026InternVL-U: Democratizing Unified Multimodal Models for Understanding, Reasoning, Generation and Editingcited by 23
- 2026BilliardPhys-Bench: Benchmarking Physical Reasoning and Visual Dynamics of Multimodal LLMscited by 1
- 2026SCI-PRM: A Tool Aware Process Reward Model for Scientific Reasoning Verificationcited by 1
- 2026ReCrit: Transition-Aware Reinforcement Learning for Scientific Critic Reasoningcited by 0
- 2026SciOrch: Learning to Orchestrate Expert LLMs for Solving Frontier Multimodal Scientific Reasoning Taskscited by 0
- 2026Ideas Have Genomes: Benchmarking Scientific Lineage Reasoning and Lineage-Grounded Idea Generationcited by 0
- 2026Ideas Have Genomes: Benchmarking Scientific Lineage Reasoning and Lineage-Grounded Idea Generation9 Jul 2026
- 2026SciOrch: Learning to Orchestrate Expert LLMs for Solving Frontier Multimodal Scientific Reasoning Tasks14 Jun 2026
- 2026SCI-PRM: A Tool Aware Process Reward Model for Scientific Reasoning Verification3 Jun 2026
- 2026BilliardPhys-Bench: Benchmarking Physical Reasoning and Visual Dynamics of Multimodal LLMs29 May 2026
- 2026ReCrit: Transition-Aware Reinforcement Learning for Scientific Critic Reasoning11 May 2026
- 2026InternVL-U: Democratizing Unified Multimodal Models for Understanding, Reasoning, Generation and Editing10 Mar 2026
- 2026CreativeBench: Benchmarking and Enhancing Machine Creativity via Self-Evolving Challengescited by 9
- 2026SciIntegrity-Bench: A Benchmark for Evaluating Academic Integrity in AI Scientist Systemscited by 4
- 2025SciEvalKit: An Open-source Evaluation Toolkit for Scientific General Intelligencecited by 3
- 2026Evaluating the Progression of Large Language Model Capabilities for Small-Molecule Drug Designcited by 3
- 2026HarmonyCell: Automating Single-Cell Perturbation Modeling under Semantic and Distribution Shiftscited by 2
- 2026FlowPIE: Test-Time Scientific Idea Evolution with Flow-Guided Literature Explorationcited by 2
- 2026Beyond Gemini-3-Pro: Revisiting LLM Routing and Aggregation at Scalecited by 1
- 2026AblateCell: A Reproduce-then-Ablate Agent for Virtual Cell Repositoriescited by 0
- 2026MindFlow: Mind Supernet Powered Thinking Flows for Research Idea Innovationcited by 0
- 2026MindFlow: Mind Supernet Powered Thinking Flows for Research Idea Innovation8 Oct 2026
- 2026SciIntegrity-Bench: A Benchmark for Evaluating Academic Integrity in AI Scientist Systems11 May 2026
- 2026AblateCell: A Reproduce-then-Ablate Agent for Virtual Cell Repositories21 Apr 2026
- 2026Evaluating the Progression of Large Language Model Capabilities for Small-Molecule Drug Design17 Apr 2026
- 2026FlowPIE: Test-Time Scientific Idea Evolution with Flow-Guided Literature Exploration31 Mar 2026
- 2026CreativeBench: Benchmarking and Enhancing Machine Creativity via Self-Evolving Challenges12 Mar 2026
- 2026HarmonyCell: Automating Single-Cell Perturbation Modeling under Semantic and Distribution Shifts2 Mar 2026
- 2026Beyond Gemini-3-Pro: Revisiting LLM Routing and Aggregation at Scale4 Jan 2026
- 2025SciEvalKit: An Open-source Evaluation Toolkit for Scientific General Intelligence26 Dec 2025
Similar, not cited
Close in wording, no citation either way
- 2025Towards Scientific Intelligence: A Survey of LLM-based Scientific Agents
- 2026AI scientists produce results without reasoning scientifically
- 2025Demystifying Scientific Problem-Solving in LLMs by Probing Knowledge and Reasoning
- 2026Beyond the Answer: Decoding the Behavior of LLMs as Scientific Reasoners
- 2026Unlocking LLM Creativity in Science through Analogical Reasoning
- 2025Idea2Plan: Exploring AI-Powered Research Planning
- 2026SciAgentGym: Benchmarking Multi-Step Scientific Tool-use in LLM Agents
- 2025From Reasoning to Learning: A Survey on Hypothesis Discovery and Rule Learning with Large Language Models