Foundation Models for Zero-Shot Segmentation of Scientific Images without AI-Ready Data
Not on the map. Why?
Abstract (arXiv)
Zero-shot and prompt-based models have excelled at visual reasoning tasks by leveraging large-scale natural image corpora, but they often fail on sparse and domain-specific scientific image data. We introduce Zenesis, a no-code interactive computer vision platform designed to reduce data readiness bottlenecks in scientific imaging workflows. Zenesis integrates lightweight multimodal adaptation for zero-shot inference on raw scientific data, human-in-the-loop refinement, and heuristic-based temporal enhancement. We validate our approach on Focused Ion Beam Scanning Electron Microscopy (FIB-SEM) datasets of catalyst-loaded membranes. Zenesis outperforms baselines, achieving an average accuracy of 0.947, Intersection over Union (IoU) of 0.858, and Dice score of 0.923 on amorphous catalyst samples; and 0.987 accuracy, 0.857 IoU, and 0.923 Dice on crystalline samples. These results represent a significant performance gain over conventional methods such as Otsu thresholding and standalone models like the Segment Anything Model (SAM). Zenesis enables effective image segmentation in domains where annotated datasets are limited, offering a scalable solution for scientific discovery.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 17 of its 39 references that are in the map. First posted on arXiv 30 Jun 2025.
Builds on
17 of 39 references.
References · 17
- 2023Segment Anythingcited by 8,158
- 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
- 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,274
- 2024SAM 2: Segment Anything in Images and Videoscited by 2,784
- 20163D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotationcited by 1,471
- 2024Grounded SAM: Assembling Open-World Models for Diverse Visual Taskscited by 914
- 2021Image Segmentation Using Text and Image Promptscited by 509
- 2023Faster Segment Anything: Towards Lightweight SAM for Mobile Applicationscited by 345
- 2023Segment Anything Model for Medical Image Analysis: an Experimental Studycited by 291
- 2023Fast Segment Anythingcited by 286
- 2017DeepIGeoS: A Deep Interactive Geodesic Framework for Medical Image Segmentationcited by 110
- 2017Interactive Medical Image Segmentation using Deep Learning with Image-specific Fine-tuningcited by 110
5 more
- 2024VISTA3D: A Unified Segmentation Foundation Model For 3D Medical Imagingcited by 62
- 2024Data Readiness for AI: A 360-Degree Surveycited by 15
- 2024MedUHIP: Towards Human-In-the-Loop Medical Segmentationcited by 5
- 2024Adapting Segment Anything Model (SAM) to Experimental Datasets via Fine-Tuning on GAN-based Simulation: A Case Study in Additive Manufacturingcited by 4
- 2024Parameter-Efficient Quantized Mixture-of-Experts Meets Vision-Language Instruction Tuning for Semiconductor Electron Micrograph Analysiscited by 1
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026Bridging Foundation Models and ASTM Metallurgical Standards for Automated Grain Size Estimation from Microscopy Images
- 2026Camyla: Scaling Autonomous Research in Medical Image Segmentation
- 2024MMSci: A Dataset for Graduate-Level Multi-Discipline Multimodal Scientific Understanding
- 2026STEMGym: Benchmarking Sequential Decision-Making under Dose Budgets in Autonomous Electron Microscopy
- 2024Learning Zero-Shot Material States Segmentation, by Implanting Natural Image Patterns in Synthetic Data
- 2024Zero-Shot Distillation for Image Encoders: How to Make Effective Use of Synthetic Data
- 2024Foundational Model for Electron Micrograph Analysis: Instruction-Tuning Small-Scale Language-and-Vision Assistant for Enterprise Adoption
- 2026A Mixture of Experts Foundation Model for Scanning Electron Microscopy Image Analysis