Science Edge Evaluation: SEE the Missing Step Toward Real Scientific Discovery
Not on the map. Why?
Abstract (arXiv)
Large language models (LLMs) are increasingly involved in scientific discovery, yet it remains unclear whether they can support complex real laboratory science. Here we introduce Science Edge Evaluation (SEE), a multimodal benchmark of expert-curated questions grounded in peer-reviewed literature and experimental practice in chemistry, biology, and materials science. Evaluation of 19 multimodal large language models (MLLMs) shows that even the best-performing model reaches only 48.7% accuracy. Moreover, general-purpose models outperform science-specialized models on average. In the visual-agent evaluation, the use of tools increases the best accuracy to 52.7%. Tool use can expand the information available to models, but more information does not necessarily lead to reliable scientific reasoning. The key challenge is whether models can manage tool-derived information within the boundaries of the original experimental evidence. Together, these findings reveal that current MLLMs still cannot reliably make justified and evidence-bounded inferences from experimental results, which is an essential capability in real scientific discovery. Bridging this gap requires MLLMs to transition from explaining established scientific concepts to deriving novel and evidence-based insights from experimental data.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 38 of its 58 references that are in the map. First posted on arXiv 7 Aug 2026.
Builds on
38 of 58 references.
References · 38
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
- 2020Measuring Massive Multitask Language Understandingcited by 7,963
- 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,879
- 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,080
- 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answeringcited by 1,919
- 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,848
- 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,614
- 2023LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Daycited by 1,305
- 2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilitiescited by 1,239
- 2024OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problemscited by 1,136
- 2024The AI Scientist: Towards Fully Automated Open-Ended Scientific Discoverycited by 1,071
- 2022Galactica: A Large Language Model for Sciencecited by 744
26 more
- 2023ChemCrow: Augmenting large-language models with chemistry toolscited by 631
- 2022BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Miningcited by 538
- 2024MMMU-Pro: A More Robust Multi-discipline Multimodal Understanding Benchmarkcited by 512
- 2023CMMLU: Measuring massive multitask language understanding in Chinesecited by 449
- 2025Agent Laboratory: Using LLM Agents as Research Assistantscited by 425
- 2025Accelerating scientific discovery with Co-Scientistcited by 401
- 2023Towards Generalist Biomedical AIcited by 289
- 2023MEDITRON-70B: Scaling Medical Pretraining for Large Language Modelscited by 274
- 2023SciBench: Evaluating College-Level Scientific Problem-Solving Abilities of Large Language Modelscited by 256
- 2023A Study of Generative Large Language Model for Medical Research and Healthcarecited by 163
- 2024LAB-Bench: Measuring Capabilities of Language Models for Biology Researchcited by 156
- 2023Mol-Instructions: A Large-Scale Biomolecular Instruction Dataset for Large Language Modelscited by 133
- 2024LlaSMol: Advancing Large Language Models for Chemistry with a Large-Scale, Comprehensive, High-Quality Instruction Tuning Datasetcited by 128
- 2023SciEval: A Multi-Level Large Language Model Evaluation Benchmark for Scientific Researchcited by 115
- 2024ChemLLM: A Chemical Large Language Modelcited by 106
- 2023BioMedGPT: Open Multimodal Generative Pre-trained Transformer for BioMedicinecited by 100
- 2024BioMedLM: A 2.7B Parameter Language Model Trained On Biomedical Textcited by 86
- 2024Developing ChemDFM as a large language foundation model for chemistrycited by 52
- 2024SciFIBench: Benchmarking Large Multimodal Models for Scientific Figure Interpretationcited by 48
- 2024Probing the limitations of multimodal language models for chemistry and materials researchcited by 32
- 2023HoneyBee: Progressive Instruction Finetuning of Large Language Models for Materials Sciencecited by 22
- 2025MMSciBench: Benchmarking Language Models on Chinese Multimodal Scientific Problemscited by 10
- 2025MME-SCI: A Comprehensive and Challenging Science Benchmark for Multimodal Large Language Modelscited by 6
- 2025Evaluating Large Language Models on Multimodal Chemistry Olympiad Examscited by 6
- 2026S1-VL: Scientific Multimodal Reasoning Model with Thinking-with-Imagescited by 5
- 2025ChemVTS-Bench: Evaluating Visual-Textual-Symbolic Reasoning of Multimodal Large Language Models in Chemistrycited by 1
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026From Knowledge to Action: Outcomes of the 2025 Large Language Model (LLM) Hackathon for Applications in Materials Science and Chemistry
- 2024SciKnowEval: Evaluating Multi-level Scientific Knowledge of Large Language Models
- 2025Transforming Science with Large Language Models: A Survey on AI-assisted Scientific Discovery, Experimentation, Content Generation, and Evaluation
- 2024ScienceAgentBench: Toward Rigorous Assessment of Language Agents for Data-Driven Scientific Discovery
- 2025RxnBench: A Multimodal Benchmark for Evaluating Large Language Models on Chemical Reaction Understanding from Scientific Literature
- 2023Large Language Models are Zero Shot Hypothesis Proposers
- 2024MOOSE-Chem: Large Language Models for Rediscovering Unseen Chemistry Scientific Hypotheses
- 2025CURIE: Evaluating LLMs On Multitask Scientific Long Context Understanding and Reasoning