S1-Omni-Image: A Unified Model for Scientific Image Understanding, Generation, and Editing
Not on the map. Why?
Abstract (arXiv)
We present S1-Omni-Image, an open-weight unified multimodal model for scientific image understanding, generation, and editing. Unlike general-purpose image generation models, scientific image tasks require not only high-fidelity synthesis, but also robust understanding of scientific semantics, structural relations, domain knowledge, and task intent. To this end, S1-Omni-Image builds on the scientific multimodal reasoning backbone S1-VL-32B and couples its understanding capability with an image generation module under a unified think-before-generate paradigm. Given a user instruction, the model first produces a task-oriented reasoning trace, a textual answer, and a task special token; their hidden states are then injected into the generation module to condition image generation or editing. S1-Omni-Image supports scientific image understanding, generation, and editing in a unified framework. For generation, it focuses on scientific illustrations and text rendering, including logical diagrams, relational comparisons, data charts, and realistic scientific visualizations. For editing, it casts segmentation and other domain-specific vision tasks as native image editing problems, enabling multi-turn illustration editing, medical and geographic image segmentation, medical image translation, and scientific image super-resolution. We construct SciGenEdit, a 314K-sample training dataset, and release the model weights, inference code, and SciGenEdit-10K. Experiments show that S1-Omni-Image substantially improves scientific image generation and editing while preserving the scientific image understanding capability inherited from S1-VL-32B. It outperforms open-source models on GenExam and TechImage-Bench, achieves state-of-the-art results on four editing benchmarks including MSD, cigRockSEM, SynthRAD2025, and IXI, and maintains stable performance on scientific image understanding evaluations.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 41 of its 65 references that are in the map. First posted on arXiv 23 Jun 2026.
Builds on
41 of 65 references.
References · 41
- 2020Denoising Diffusion Probabilistic Modelscited by 20,851
- 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
- 2023Visual Instruction Tuningcited by 7,792
- 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
- 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,510
- 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
- 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
- 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,591
- 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
- 2025Qwen3-VL Technical Reportcited by 2,534
- 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,409
29 more
- 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,044
- 2025Qwen-Image Technical Reportcited by 973
- 2025OpenAI GPT-5 System Cardcited by 910
- 2025Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scalingcited by 787
- 2024Show-o: One Single Transformer to Unify Multimodal Understanding and Generationcited by 734
- 2022DiffEdit: Diffusion-based semantic image editing with mask guidancecited by 643
- 2021The Medical Segmentation Decathloncited by 620
- 2024Measuring Multimodal Mathematical Reasoning with MATH-Vision Datasetcited by 516
- 2024Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and Generationcited by 447
- 2023Generative Multimodal Models are In-Context Learnerscited by 446
- 2021Self-Supervised Pre-Training of Swin Transformers for 3D Medical Image Analysiscited by 418
- 2025Step1X-Edit: A Practical Framework for General Image Editingcited by 417
- 2024SWIFT:A Scalable lightWeight Infrastructure for Fine-Tuningcited by 345
- 2024SEED-X: Multimodal Models with Unified Multi-granularity Comprehension and Generationcited by 328
- 2025Thinking with Images for Multimodal Reasoning: Foundations, Methods, and Future Frontierscited by 193
- 2025Thyme: Think Beyond Imagescited by 160
- 2024Divide, Conquer and Combine: A Training-Free Framework for High-Resolution Image Perception in Multimodal Large Language Modelscited by 151
- 2024GMAI-MMBench: A Comprehensive Multimodal Evaluation Benchmark Towards General Medical AIcited by 96
- 2023AutomaTikZ: Text-Guided Synthesis of Scientific Vector Graphics with TikZcited by 74
- 2022BCI: Breast Cancer Immunohistochemical Image Generation through Pyramid Pix2pixcited by 61
- 2025Intern-S1: A Scientific Multimodal Foundation Modelcited by 49
- 2025Scientists' First Exam: Probing Cognitive Abilities of MLLM via Perception, Understanding, and Reasoningcited by 33
- 2025SynthRAD2025 Grand Challenge dataset: generating synthetic CTs for radiotherapycited by 27
- 2025TikZero: Zero-Shot Text-Guided Graphics Program Synthesiscited by 21
- 2024Improving Long-Text Alignment for Text-to-Image Diffusion Modelscited by 18
- 2026S1-VL: Scientific Multimodal Reasoning Model with Thinking-with-Imagescited by 5
- 2021What Makes a Scientific Paper be Accepted for Publication?cited by 4
- 2025Virtual Full-stack Scanning of Brain MRI via Imputing Any Quantised Codecited by 4
- 2025Why Registration Quality Matters: Enhancing sCT Synthesis with IMPACT-Based Registrationcited by 4
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2026UniReason 1.0: A Unified Reasoning Framework for World Knowledge Aligned Image Generation and Editing
- 2026Omni-R1: Towards the Unified Generative Paradigm for Multimodal Reasoning
- 2026SciGen-Verifier: A Multimodal Reasoner for Explainable Verification in Scientific Image Generation
- 2025Uni-MMMU: A Massive Multi-discipline Multimodal Unified Benchmark
- 2026Scientific Image Synthesis: Benchmarking, Methodologies, and Downstream Utility
- 2025GoT: Unleashing Reasoning Capability of Multimodal Large Language Model for Visual Generation and Editing
- 2026SciIR: A Large-scale Training Dataset and Benchmark for Scientific Image Reasoning Generation
- 2026GRADE: Benchmarking Discipline-Informed Reasoning in Image Editing