mlsanityData to 7 Oct 2026

OmniScientist: An Omni-Modal Omni-Discipline AI Scientist

arXiv 2608.13558 ↗First posted 13 Aug 2026cs.AI cs.CLcited by 1

Not on the map. Why?

Abstract (arXiv)

Recent advances in foundation models have enabled AI scientists to automate increasingly complete research workflows, from hypothesis generation and code execution to manuscript preparation. Yet workflow coverage alone does not provide access to the full evidence on which scientific discovery depends. Existing systems typically reason over text, code, labels, or precomputed summaries, leaving scientifically decisive spatial, temporal, cross-channel, and procedural relations unavailable to the agent. We introduce OmniScientist, an end-to-end, omni-modal AI scientist that conducts multidisciplinary research directly from heterogeneous raw evidence. A perception layer and 3 autonomous agents for ideation, experiment, and writeup operate within a deterministic pipeline, allowing observations to shape research questions, experimental decisions, and final claims throughout the research lifecycle. By running idea, rigour, and claim checks in code, the system enforces novelty screening, statistical validity, execution provenance, and numerical traceability. We evaluate OmniScientist on 36 real-data cases spanning 5 discipline families, 4 families of scientific evidence, and modalities including images, signals, audio, video, 3-D structures, trajectories, tables, formulae, and graphs. The system completes the full path from raw data to a compiled manuscript in all 36 cases and achieves a mean overall paper score of 6.3 with the reference reasoning backbone. In paired comparisons against a blind variant that receives only precomputed scalar features, direct perception improves all 7 evaluation dimensions and wins 85% of head-to-head judgments. These results show that lifecycle-wide perception is essential for evidence-grounded scientific discovery and provides a practical path toward broadly capable AI scientists.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 57 of its 95 references that are in the map. First posted on arXiv 13 Aug 2026.

Builds on

57 of 95 references.

References · 57

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  4. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  5. 2023Visual Instruction Tuningcited by 7,792
  6. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  7. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  8. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  9. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  10. 2020Open Graph Benchmark: Datasets for Machine Learning on Graphscited by 2,148
  11. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,044
  12. 2017EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classificationcited by 1,488
38 more
  1. 2017Multimodal Machine Learning: A Survey and Taxonomycited by 1,004
  2. 2021MedMNIST v2 -- A large-scale lightweight benchmark for 2D and 3D biomedical image classificationcited by 682
  3. 2019AI Feynman: a Physics-Inspired Method for Symbolic Regressioncited by 536
  4. 2023MM-REACT: Prompting ChatGPT for Multimodal Reasoning and Actioncited by 511
  5. 2024MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineeringcited by 429
  6. 2025Agent Laboratory: Using LLM Agents as Research Assistantscited by 405
  7. 2022PDEBENCH: An Extensive Benchmark for Scientific Machine Learningcited by 396
  8. 2025The AI Scientist-v2: Workshop-Level Automated Scientific Discovery via Agentic Tree Searchcited by 375
  9. 2025Accelerating scientific discovery with Co-Scientistcited by 353
  10. 2025PaperBench: Evaluating AI's Ability to Replicate AI Researchcited by 286
  11. 2024CharXiv: Charting Gaps in Realistic Chart Understanding in Multimodal LLMscited by 229
  12. 2024ScienceAgentBench: Toward Rigorous Assessment of Language Agents for Data-Driven Scientific Discoverycited by 208
  13. 2025AIDE: AI-Driven Exploration in the Space of Codecited by 206
  14. 2022OpenAlex: A fully-open index of scholarly works, authors, venues, institutions, and conceptscited by 166
  15. 2026Towards End-to-End Automation of AI Researchcited by 150
  16. 2015An open access repository of images on plant health to enable the development of mobile disease diagnosticscited by 132
  17. 2025AI-Researcher: Autonomous Scientific Innovationcited by 88
  18. 2025Kosmos: An AI Scientist for Autonomous Discoverycited by 78
  19. 2024SPIQA: A Dataset for Multimodal Question Answering on Scientific Paperscited by 75
  20. 2025From AI for Science to Agentic Science: A Survey on Autonomous Scientific Discoverycited by 74
  21. 2024ORGANA: A Robotic Assistant for Automated Chemistry Experimentation and Characterizationcited by 60
  22. 2019MIMII Dataset: Sound Dataset for Malfunctioning Industrial Machine Investigation and Inspectioncited by 57
  23. 2024SciFIBench: Benchmarking Large Multimodal Models for Scientific Figure Interpretationcited by 47
  24. 2025ScienceBoard: Evaluating Multimodal Autonomous Agents in Realistic Scientific Workflowscited by 47
  25. 2018Automatic acoustic detection of birds through deep learning: the first Bird Audio Detection challengecited by 44
  26. 2025InternAgent: When Agent Becomes the Scientist -- Building Closed-Loop System from Hypothesis to Verificationcited by 41
  27. 2021Galaxy Zoo DECaLS: Detailed Visual Morphology Measurements from Volunteers and Deep Learning for 314,000 Galaxiescited by 40
  28. 2018A Commute in Data: The comma2k19 Datasetcited by 33
  29. 2024NatureLM-audio: an Audio-Language Foundation Model for Bioacousticscited by 32
  30. 2021The Multi-Agent Behavior Dataset: Mouse Dyadic Social Interactionscited by 31
  31. 2024Probing the limitations of multimodal language models for chemistry and materials researchcited by 31
  32. 2025MicroVQA: A Multimodal Reasoning Benchmark for Microscopy-Based Scientific Researchcited by 30
  33. 2022The Caltech Fish Counting Dataset: A Benchmark for Multiple-Object Tracking and Countingcited by 24
  34. 2024MMSci: A Dataset for Graduate-Level Multi-Discipline Multimodal Scientific Understandingcited by 23
  35. 2025Dolphin: Moving Towards Closed-loop Auto-research through Thinking, Practice, and Feedbackcited by 23
  36. 2025The Denario project: Deep knowledge AI agents for scientific discoverycited by 21
  37. 2015WHOI-Plankton- A Large Scale Fine Grained Visual Recognition Benchmark Dataset for Plankton Classificationcited by 17
  38. 2024μ-Bench: A Vision-Language Benchmark for Microscopy Understandingcited by 15

50 most cited of 57 All 57 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.