mlsanityData to 14 Oct 2026

Science Edge Evaluation: SEE the Missing Step Toward Real Scientific Discovery

arXiv 2608.06931 ↗First posted 7 Aug 2026cs.AIcited by 1

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) are increasingly involved in scientific discovery, yet it remains unclear whether they can support complex real laboratory science. Here we introduce Science Edge Evaluation (SEE), a multimodal benchmark of expert-curated questions grounded in peer-reviewed literature and experimental practice in chemistry, biology, and materials science. Evaluation of 19 multimodal large language models (MLLMs) shows that even the best-performing model reaches only 48.7% accuracy. Moreover, general-purpose models outperform science-specialized models on average. In the visual-agent evaluation, the use of tools increases the best accuracy to 52.7%. Tool use can expand the information available to models, but more information does not necessarily lead to reliable scientific reasoning. The key challenge is whether models can manage tool-derived information within the boundaries of the original experimental evidence. Together, these findings reveal that current MLLMs still cannot reliably make justified and evidence-bounded inferences from experimental results, which is an essential capability in real scientific discovery. Bridging this gap requires MLLMs to transition from explaining established scientific concepts to deriving novel and evidence-based insights from experimental data.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 38 of its 58 references that are in the map. First posted on arXiv 7 Aug 2026.

Builds on

38 of 58 references.

References · 38

  1. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  2. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  3. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,879
  4. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,080
  5. 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answeringcited by 1,919
  6. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,848
  7. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,614
  8. 2023LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Daycited by 1,305
  9. 2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilitiescited by 1,239
  10. 2024OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problemscited by 1,136
  11. 2024The AI Scientist: Towards Fully Automated Open-Ended Scientific Discoverycited by 1,071
  12. 2022Galactica: A Large Language Model for Sciencecited by 744
26 more
  1. 2023ChemCrow: Augmenting large-language models with chemistry toolscited by 631
  2. 2022BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Miningcited by 538
  3. 2024MMMU-Pro: A More Robust Multi-discipline Multimodal Understanding Benchmarkcited by 512
  4. 2023CMMLU: Measuring massive multitask language understanding in Chinesecited by 449
  5. 2025Agent Laboratory: Using LLM Agents as Research Assistantscited by 425
  6. 2025Accelerating scientific discovery with Co-Scientistcited by 401
  7. 2023Towards Generalist Biomedical AIcited by 289
  8. 2023MEDITRON-70B: Scaling Medical Pretraining for Large Language Modelscited by 274
  9. 2023SciBench: Evaluating College-Level Scientific Problem-Solving Abilities of Large Language Modelscited by 256
  10. 2023A Study of Generative Large Language Model for Medical Research and Healthcarecited by 163
  11. 2024LAB-Bench: Measuring Capabilities of Language Models for Biology Researchcited by 156
  12. 2023Mol-Instructions: A Large-Scale Biomolecular Instruction Dataset for Large Language Modelscited by 133
  13. 2024LlaSMol: Advancing Large Language Models for Chemistry with a Large-Scale, Comprehensive, High-Quality Instruction Tuning Datasetcited by 128
  14. 2023SciEval: A Multi-Level Large Language Model Evaluation Benchmark for Scientific Researchcited by 115
  15. 2024ChemLLM: A Chemical Large Language Modelcited by 106
  16. 2023BioMedGPT: Open Multimodal Generative Pre-trained Transformer for BioMedicinecited by 100
  17. 2024BioMedLM: A 2.7B Parameter Language Model Trained On Biomedical Textcited by 86
  18. 2024Developing ChemDFM as a large language foundation model for chemistrycited by 52
  19. 2024SciFIBench: Benchmarking Large Multimodal Models for Scientific Figure Interpretationcited by 48
  20. 2024Probing the limitations of multimodal language models for chemistry and materials researchcited by 32
  21. 2023HoneyBee: Progressive Instruction Finetuning of Large Language Models for Materials Sciencecited by 22
  22. 2025MMSciBench: Benchmarking Language Models on Chinese Multimodal Scientific Problemscited by 10
  23. 2025MME-SCI: A Comprehensive and Challenging Science Benchmark for Multimodal Large Language Modelscited by 6
  24. 2025Evaluating Large Language Models on Multimodal Chemistry Olympiad Examscited by 6
  25. 2026S1-VL: Scientific Multimodal Reasoning Model with Thinking-with-Imagescited by 5
  26. 2025ChemVTS-Bench: Evaluating Visual-Textual-Symbolic Reasoning of Multimodal Large Language Models in Chemistrycited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2026From Knowledge to Action: Outcomes of the 2025 Large Language Model (LLM) Hackathon for Applications in Materials Science and Chemistry
  2. 2024SciKnowEval: Evaluating Multi-level Scientific Knowledge of Large Language Models
  3. 2025Transforming Science with Large Language Models: A Survey on AI-assisted Scientific Discovery, Experimentation, Content Generation, and Evaluation
  4. 2024ScienceAgentBench: Toward Rigorous Assessment of Language Agents for Data-Driven Scientific Discovery
  5. 2025RxnBench: A Multimodal Benchmark for Evaluating Large Language Models on Chemical Reaction Understanding from Scientific Literature
  6. 2023Large Language Models are Zero Shot Hypothesis Proposers
  7. 2024MOOSE-Chem: Large Language Models for Rediscovering Unseen Chemistry Scientific Hypotheses
  8. 2025CURIE: Evaluating LLMs On Multitask Scientific Long Context Understanding and Reasoning