mlsanityData to 14 Oct 2026

A unified multimodal understanding and generation model for cross-disciplinary scientific research

arXiv 2601.01363 ↗First posted 4 Jan 2026cs.AIcited by 1

Not on the map. Why?

Abstract (arXiv)

Scientific discovery increasingly relies on integrating heterogeneous, high-dimensional data across disciplines nowadays. While AI models have achieved notable success across various scientific domains, they typically remain domain-specific or lack the capability of simultaneously understanding and generating multimodal scientific data, particularly for high-dimensional data. Yet, many pressing global challenges and scientific problems are inherently cross-disciplinary and require coordinated progress across multiple fields. Here, we present FuXi-Uni, a native unified multimodal model for scientific understanding and high-fidelity generation across scientific domains within a single architecture. Specifically, FuXi-Uni aligns cross-disciplinary scientific tokens within natural language tokens and employs science decoder to reconstruct scientific tokens, thereby supporting both natural language conversation and scientific numerical prediction. Empirically, we validate FuXi-Uni in Earth science and Biomedicine. In Earth system modeling, the model supports global weather forecasting, tropical cyclone (TC) forecast editing, and spatial downscaling driven by only language instructions. FuXi-Uni generates 10-day global forecasts at 0.25deg resolution that outperform the SOTA physical forecasting system. It shows superior performance for both TC track and intensity prediction relative to the SOTA physical model, and generates high-resolution regional weather fields that surpass standard interpolation baselines. Regarding biomedicine, FuXi-Uni outperforms leading multimodal large language models on multiple biomedical visual question answering benchmarks. By unifying heterogeneous scientific modalities within a native shared latent space while maintaining strong domain-specific performance, FuXi-Uni provides a step forward more general-purpose, multimodal scientific models.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 42 of its 129 references that are in the map. First posted on arXiv 4 Jan 2026.

Builds on

42 of 129 references.

References · 42

  1. 2023Visual Instruction Tuningcited by 7,869
  2. 2025Qwen2.5-VL Technical Reportcited by 5,587
  3. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  4. 2020GLU Variants Improve Transformercited by 1,804
  5. 2019Root Mean Square Layer Normalizationcited by 1,480
  6. 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,346
  7. 2023LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Daycited by 1,305
  8. 2014Image Super-Resolution Using Deep Convolutional Networkscited by 1,224
  9. 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 927
  10. 2025Emerging Properties in Unified Multimodal Pretrainingcited by 897
  11. 2024Emu3: Next-Token Prediction is All You Needcited by 781
  12. 2022FourCastNet: A Global Data-driven High-resolution Weather Model using Adaptive Fourier Neural Operatorscited by 776
30 more
  1. 2024Show-o: One Single Transformer to Unify Multimodal Understanding and Generationcited by 752
  2. 2022Galactica: A Large Language Model for Sciencecited by 744
  3. 2024Transfusion: Predict the Next Token and Diffuse Images with One Multi-Modal Modelcited by 483
  4. 2024Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and Generationcited by 457
  5. 2021SLAKE: A Semantically-Labeled Knowledge-Enhanced Dataset for Medical Visual Question Answeringcited by 449
  6. 2023Generative Multimodal Models are In-Context Learnerscited by 446
  7. 2020WeatherBench: A benchmark dataset for data-driven weather forecastingcited by 275
  8. 2023FuXi: A cascade machine learning forecasting system for 15-day global weather forecastcited by 241
  9. 2023Neural General Circulation Models for Weather and Climatecited by 239
  10. 2023WeatherBench 2: A benchmark for the next generation of data-driven global weather modelscited by 236
  11. 2023Guiding Instruction-based Image Editing via Multimodal Large Language Modelscited by 185
  12. 2024JanusFlow: Harmonizing Autoregression and Rectified Flow for Unified Multimodal Understanding and Generationcited by 163
  13. 2021Medical Visual Question Answering: A Surveycited by 93
  14. 2025Unified Multimodal Understanding and Generation Models: Advances, Challenges, and Opportunitiescited by 85
  15. 2017DeepSD: Generating High Resolution Climate Change Projections through Single Image Super-Resolutioncited by 82
  16. 2024MedTrinity-25M: A Large-scale Multimodal Dataset with Multigranular Annotations for Medicinecited by 81
  17. 2022Deep Learning in Multimodal Remote Sensing Data Fusion: A Comprehensive Reviewcited by 73
  18. 2021Multimodal Image Synthesis and Editing: The Generative AI Eracited by 52
  19. 2025Transforming Science with Large Language Models: A Survey on AI-assisted Scientific Discovery, Experimentation, Content Generation, and Evaluationcited by 42
  20. 2023DARWIN Series: Domain Specific Large Language Models for Natural Sciencecited by 39
  21. 2025A Survey of Scientific Large Language Models: From Data Foundations to Agent Frontierscited by 37
  22. 2024Fast, Scale-Adaptive, and Uncertainty-Aware Downscaling of Earth System Model Fields with Generative Machine Learningcited by 27
  23. 2017Intercomparison of Machine Learning Methods for Statistical Downscaling: The Case of Daily and Extreme Precipitationcited by 24
  24. 2023FuXi-Extreme: Improving extreme rainfall and wind forecasts with diffusion modelcited by 22
  25. 2025OmniScience: A Domain-Specialized LLM for Scientific Reasoning and Discoverycited by 20
  26. 2023Interpretable Medical Image Visual Question Answering via Multi-Modal Relationship Graph Learningcited by 17
  27. 2023When Geoscience Meets Foundation Models: Towards General Geoscience Artificial Intelligence Systemcited by 13
  28. 2025Discrete Tokenization for Multimodal LLMs: A Comprehensive Surveycited by 8
  29. 2024SciDFM: A Large Language Model with Mixture-of-Experts for Sciencecited by 7
  30. 2024UniMat: Unifying Materials Embeddings through Multi-modal Learningcited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2026S1-Omni: A Unified Multimodal Reasoning Model for Scientific Understanding, Prediction, and Generation
  2. 2024ChatSR: Multimodal Large Language Models for Scientific Formula Discovery
  3. 2025SciTS: Scientific Time Series Understanding and Generation with LLMs
  4. 2026Innovator-VL: A Multimodal Large Language Model for Scientific Discovery
  5. 2025SciReasoner: Laying the Scientific Reasoning Ground Across Disciplines
  6. 2025MSEarth: A Multimodal Benchmark for Earth Science Phenomenon Discovery with MLLMs
  7. 2026EO-Agents: A Three-Agent LLM Pipeline for Earth Observation Hypothesis Generation
  8. 2024INDUS: Effective and Efficient Language Models for Scientific Applications