Industrial Synthetic Segment Pre-training
Not on the map. Why?
Abstract (arXiv)
Vision Foundation Models (VFMs) have made remarkable progress and are increasingly being applied to segmentation tasks in real-world industrial settings. However, VFMs pre-trained on real-image datasets still face several challenges: (1) they do not always perform well on industrial datasets due to significant differences from natural imagery, (2) legal and ethical restrictions, such as limitations on commercial use, constrain extensibility, and (3) building training frameworks under limited computational and data resources remains a critical issue. These challenges raise a fundamental question: can we construct industrial segmentation models without relying on real images or manual annotations? To address this question, we propose the Instance Core Segment Dataset (InsCore), a synthetic data generation framework and the resulting pre-training dataset based on Formula-Driven Supervised Learning (FDSL). InsCore is designed not around the visual appearance or domain of real images, but around the hypothesis that learning to handle complex occlusions during pre-training is a key factor for strong performance in industrial domains. Through experiments across five domains (medical, biomedical, remote sensing, manufacturing, and logistics) we demonstrate that InsCore pre-trained models achieve average mAP scores of 45.2 with the ViTDet backbone and 46.0 with the Swin Transformer backbone, on par with ImageNet-21k supervised pre-training (45.0) while using no real images at all. As a reference point under different input assumptions, prompted SAM with ground-truth bounding boxes attains 45.4 on the same benchmarks. Finally, InsCore consists of only 100k images and 3.2M masks, roughly 1/110 and 1/312 the scale of the SA-1B dataset.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 43 of its 55 references that are in the map. First posted on arXiv 19 May 2025.
Builds on
43 of 55 references.
References · 43
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
- 2014Microsoft COCO: Common Objects in Contextcited by 17,608
- 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
- 2023Segment Anythingcited by 8,234
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
- 2017Mask R-CNNcited by 7,277
- 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
- 2024SAM 2: Segment Anything in Images and Videoscited by 2,863
- 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,351
- 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,498
- 2019MMDetection: Open MMLab Detection Toolbox and Benchmarkcited by 1,385
31 more
- 2021Scaling Vision Transformerscited by 999
- 2017Revisiting Unreasonable Effectiveness of Data in Deep Learning Eracited by 919
- 2022Exploring Plain Vision Transformer Backbones for Object Detectioncited by 724
- 2020Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understandingcited by 604
- 2019YOLACT: Real-time Instance Segmentationcited by 443
- 2013DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognitioncited by 424
- 2020Virtual KITTI 2cited by 353
- 2018Training Deep Networks with Synthetic Data: Bridging the Reality Gap by Domain Randomizationcited by 294
- 2021SynthSeg: Segmentation of brain MRI scans of any contrast and resolution without retrainingcited by 237
- 2022RTMDet: An Empirical Study of Designing Real-Time Object Detectorscited by 195
- 2018SpaceNet: A Remote Sensing Dataset and Challenge Seriescited by 179
- 2023StableRep: Synthetic Images from Text-to-Image Models Make Strong Visual Representation Learnerscited by 177
- 2019Towards Fairer Datasets: Filtering and Balancing the Distribution of the People Subtree in the ImageNet Hierarchycited by 156
- 2023Segment Anything Is Not Always Perfect: An Investigation of SAM on Different Real-world Applicationscited by 121
- 2018Learning Semantic Segmentation from Synthetic Data: A Geometrically Guided Input-Output Adaptation Approachcited by 119
- 2021Pre-training without Natural Imagescited by 100
- 2023The Segment Anything Model (SAM) for Remote Sensing Applications: From Zero to One Shotcited by 83
- 2023Into the LAIONs Den: Investigating Hate in Multimodal Datasetscited by 54
- 2023Computer-Vision Benchmark Segment-Anything Model (SAM) in Medical Images: Accuracy in 12 Datasetscited by 45
- 2021Can Vision Transformers Learn without Natural Images?cited by 32
- 2021Task2Sim : Towards Effective Pre-training and Transfer from Synthetic Datacited by 26
- 2019Adversarial Learning and Self-Teaching Techniques for Domain Adaptation in Semantic Segmentationcited by 22
- 2022Pre-training Vision Transformers with Formula-driven Supervised Learningcited by 21
- 2023Visual Atoms: Pre-training Vision Transformers with Sinusoidal Wavescited by 20
- 2023SegGen: Supercharging Segmentation Models with Text2Mask and Mask2Img Synthesiscited by 18
- 2024Scaling Backwards: Minimal Synthetic Pre-training?cited by 14
- 2024Low-Resource Vision Challenges for Foundation Modelscited by 13
- 2023Pre-training Vision Transformers with Very Limited Synthesized Imagescited by 12
- 2023SegRCDB: Semantic Segmentation via Formula-Driven Supervised Learningcited by 8
- 2024TS-SAM: Fine-Tuning Segment-Anything Model for Downstream Taskscited by 5
- 2024Formula-Supervised Visual-Geometric Pre-trainingcited by 3
Built on by
Not cited yet.