mlsanityData to 7 Oct 2026

S1-Omni-Image: A Unified Model for Scientific Image Understanding, Generation, and Editing

arXiv 2606.24441 ↗First posted 23 Jun 2026cs.CVcited by 2

Not on the map. Why?

Abstract (arXiv)

We present S1-Omni-Image, an open-weight unified multimodal model for scientific image understanding, generation, and editing. Unlike general-purpose image generation models, scientific image tasks require not only high-fidelity synthesis, but also robust understanding of scientific semantics, structural relations, domain knowledge, and task intent. To this end, S1-Omni-Image builds on the scientific multimodal reasoning backbone S1-VL-32B and couples its understanding capability with an image generation module under a unified think-before-generate paradigm. Given a user instruction, the model first produces a task-oriented reasoning trace, a textual answer, and a task special token; their hidden states are then injected into the generation module to condition image generation or editing. S1-Omni-Image supports scientific image understanding, generation, and editing in a unified framework. For generation, it focuses on scientific illustrations and text rendering, including logical diagrams, relational comparisons, data charts, and realistic scientific visualizations. For editing, it casts segmentation and other domain-specific vision tasks as native image editing problems, enabling multi-turn illustration editing, medical and geographic image segmentation, medical image translation, and scientific image super-resolution. We construct SciGenEdit, a 314K-sample training dataset, and release the model weights, inference code, and SciGenEdit-10K. Experiments show that S1-Omni-Image substantially improves scientific image generation and editing while preserving the scientific image understanding capability inherited from S1-VL-32B. It outperforms open-source models on GenExam and TechImage-Bench, achieves state-of-the-art results on four editing benchmarks including MSD, cigRockSEM, SynthRAD2025, and IXI, and maintains stable performance on scientific image understanding evaluations.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 41 of its 65 references that are in the map. First posted on arXiv 23 Jun 2026.

Builds on

41 of 65 references.

References · 41

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2023Visual Instruction Tuningcited by 7,792
  5. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  6. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,510
  7. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  8. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
  9. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,591
  10. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
  11. 2025Qwen3-VL Technical Reportcited by 2,534
  12. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,409
29 more
  1. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,044
  2. 2025Qwen-Image Technical Reportcited by 973
  3. 2025OpenAI GPT-5 System Cardcited by 910
  4. 2025Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scalingcited by 787
  5. 2024Show-o: One Single Transformer to Unify Multimodal Understanding and Generationcited by 734
  6. 2022DiffEdit: Diffusion-based semantic image editing with mask guidancecited by 643
  7. 2021The Medical Segmentation Decathloncited by 620
  8. 2024Measuring Multimodal Mathematical Reasoning with MATH-Vision Datasetcited by 516
  9. 2024Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and Generationcited by 447
  10. 2023Generative Multimodal Models are In-Context Learnerscited by 446
  11. 2021Self-Supervised Pre-Training of Swin Transformers for 3D Medical Image Analysiscited by 418
  12. 2025Step1X-Edit: A Practical Framework for General Image Editingcited by 417
  13. 2024SWIFT:A Scalable lightWeight Infrastructure for Fine-Tuningcited by 345
  14. 2024SEED-X: Multimodal Models with Unified Multi-granularity Comprehension and Generationcited by 328
  15. 2025Thinking with Images for Multimodal Reasoning: Foundations, Methods, and Future Frontierscited by 193
  16. 2025Thyme: Think Beyond Imagescited by 160
  17. 2024Divide, Conquer and Combine: A Training-Free Framework for High-Resolution Image Perception in Multimodal Large Language Modelscited by 151
  18. 2024GMAI-MMBench: A Comprehensive Multimodal Evaluation Benchmark Towards General Medical AIcited by 96
  19. 2023AutomaTikZ: Text-Guided Synthesis of Scientific Vector Graphics with TikZcited by 74
  20. 2022BCI: Breast Cancer Immunohistochemical Image Generation through Pyramid Pix2pixcited by 61
  21. 2025Intern-S1: A Scientific Multimodal Foundation Modelcited by 49
  22. 2025Scientists' First Exam: Probing Cognitive Abilities of MLLM via Perception, Understanding, and Reasoningcited by 33
  23. 2025SynthRAD2025 Grand Challenge dataset: generating synthetic CTs for radiotherapycited by 27
  24. 2025TikZero: Zero-Shot Text-Guided Graphics Program Synthesiscited by 21
  25. 2024Improving Long-Text Alignment for Text-to-Image Diffusion Modelscited by 18
  26. 2026S1-VL: Scientific Multimodal Reasoning Model with Thinking-with-Imagescited by 5
  27. 2021What Makes a Scientific Paper be Accepted for Publication?cited by 4
  28. 2025Virtual Full-stack Scanning of Brain MRI via Imputing Any Quantised Codecited by 4
  29. 2025Why Registration Quality Matters: Enhancing sCT Synthesis with IMPACT-Based Registrationcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026UniReason 1.0: A Unified Reasoning Framework for World Knowledge Aligned Image Generation and Editing
  2. 2026Omni-R1: Towards the Unified Generative Paradigm for Multimodal Reasoning
  3. 2026SciGen-Verifier: A Multimodal Reasoner for Explainable Verification in Scientific Image Generation
  4. 2025Uni-MMMU: A Massive Multi-discipline Multimodal Unified Benchmark
  5. 2026Scientific Image Synthesis: Benchmarking, Methodologies, and Downstream Utility
  6. 2025GoT: Unleashing Reasoning Capability of Multimodal Large Language Model for Visual Generation and Editing
  7. 2026SciIR: A Large-scale Training Dataset and Benchmark for Scientific Image Reasoning Generation
  8. 2026GRADE: Benchmarking Discipline-Informed Reasoning in Image Editing