# Adapting Segment Anything Model (SAM) to Experimental Datasets via Fine-Tuning on GAN-based Simulation: A Case Study in Additive Manufacturing

arXiv: https://arxiv.org/abs/2412.11381 · HTML: https://mlsanity.com/paper/2412.11381/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 30 references that are in the map.
- First posted on arXiv 16 Dec 2024.

## Builds on (most cited first, 7 of 7)

- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer](https://mlsanity.com/paper/1701.06538/) (2017)
- [Contrastive Learning for Unpaired Image-to-Image Translation](https://mlsanity.com/paper/2007.15651/) (2020)
- [Measuring Catastrophic Forgetting in Neural Networks](https://mlsanity.com/paper/1708.02072/) (2017)
- [Segment Anything Model for Medical Image Analysis: an Experimental Study](https://mlsanity.com/paper/2304.10517/) (2023)
- [Convolution Meets LoRA: Parameter Efficient Finetuning for Segment Anything Model](https://mlsanity.com/paper/2401.17868/) (2024)
- [Automatic Volumetric Segmentation of Additive Manufacturing Defects with 3D U-Net](https://mlsanity.com/paper/2101.08993/) (2021)

## Built on by (most cited first, 5 of 5)

- [XCT-SAM: Sequential Parameter-Efficient Domain Adaptation of SAM for Industrial XCT Defect Segmentation](https://mlsanity.com/paper/2607.14287/) (2026)
- [Foundation Models for Zero-Shot Segmentation of Scientific Images without AI-Ready Data](https://mlsanity.com/paper/2506.24039/) (2025)
- [Weakly Supervised Food Image Segmentation using Vision Transformers and Segment Anything Model](https://mlsanity.com/paper/2509.19028/) (2025)
- [DCM-SAM: Defect-Conditioned Mixture of LoRA Experts for NPU-Deployed AM Defect Segmentation](https://mlsanity.com/paper/2609.38811/) (2026)
- [BaT-SAM: Introducing Inductive Bias into SAM via a Mixture of Convolutional Experts for Brain Tumor Segmentation](https://mlsanity.com/paper/2610.15330/) (2026)

## Abstract (arXiv)

Industrial X-ray computed tomography (XCT) is a powerful tool for non-destructive characterization of materials and manufactured components. XCT commonly accompanied by advanced image analysis and computer vision algorithms to extract relevant information from the images. Traditional computer vision models often struggle due to noise, resolution variability, and complex internal structures, particularly in scientific imaging applications. State-of-the-art foundational models, like the Segment Anything Model (SAM)-designed for general-purpose image segmentation-have revolutionized image segmentation across various domains, yet their application in specialized fields like materials science remains under-explored. In this work, we explore the application and limitations of SAM for industrial X-ray CT inspection of additive manufacturing components. We demonstrate that while SAM shows promise, it struggles with out-of-distribution data, multiclass segmentation, and computational efficiency during fine-tuning. To address these issues, we propose a fine-tuning strategy utilizing parameter-efficient techniques, specifically Conv-LoRa, to adapt SAM for material-specific datasets. Additionally, we leverage generative adversarial network (GAN)-generated data to enhance the training process and improve the model's segmentation performance on complex X-ray CT data. Our experimental results highlight the importance of tailored segmentation models for accurate inspection, showing that fine-tuning SAM on domain-specific scientific imaging data significantly improves performance. However, despite improvements, the model's ability to generalize across diverse datasets remains limited, highlighting the need for further research into robust, scalable solutions for domain-specific segmentation tasks.