# A Multispectral Automated Transfer Technique (MATT) for machine-driven image labeling utilizing the Segment Anything Model (SAM)

arXiv: https://arxiv.org/abs/2402.11413 · HTML: https://mlsanity.com/paper/2402.11413/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 101 references that are in the map.
- First posted on arXiv 18 Feb 2024.

## Builds on (most cited first, 23 of 23)

- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation](https://mlsanity.com/paper/1511.00561/) (2015)
- [Medical SAM Adapter: Adapting Segment Anything Model for Medical Image Segmentation](https://mlsanity.com/paper/2304.12620/) (2023)
- [Deep Multi-modal Object Detection and Semantic Segmentation for Autonomous Driving: Datasets, Methods, and Challenges](https://mlsanity.com/paper/1902.07830/) (2019)
- [Segment Anything Model for Medical Image Analysis: an Experimental Study](https://mlsanity.com/paper/2304.10517/) (2023)
- [Customized Segment Anything Model for Medical Image Segmentation](https://mlsanity.com/paper/2304.13785/) (2023)
- [The Segment Anything Model (SAM) for Remote Sensing Applications: From Zero to One Shot](https://mlsanity.com/paper/2306.16623/) (2023)
- [Learning with Limited Annotations: A Survey on Deep Semi-Supervised Learning for Medical Image Segmentation](https://mlsanity.com/paper/2207.14191/) (2022)
- [Adapting Segment Anything Model for Change Detection in HR Remote Sensing Images](https://mlsanity.com/paper/2309.01429/) (2023)
- [Segment anything, from space?](https://mlsanity.com/paper/2304.13000/) (2023)
- [Segmenting Objects in Day and Night:Edge-Conditioned CNN for Thermal Image Semantic Segmentation](https://mlsanity.com/paper/1907.10303/) (2019)
- [SAM-Assisted Remote Sensing Imagery Semantic Segmentation with Object and Boundary Constraints](https://mlsanity.com/paper/2312.02464/) (2023)
- [Glass Segmentation with RGB-Thermal Image Pairs](https://mlsanity.com/paper/2204.05453/) (2022)
- [Semantic Segmentation for Thermal Images: A Comparative Survey](https://mlsanity.com/paper/2205.13278/) (2022)
- [Tree-GPT: Modular Large Language Model Expert System for Forest Remote Sensing Image Understanding and Interactive Analysis](https://mlsanity.com/paper/2310.04698/) (2023)
- [Object Detection in Thermal Spectrum for Advanced Driver-Assistance Systems (ADAS)](https://mlsanity.com/paper/2109.09854/) (2021)
- [DooDLeNet: Double DeepLab Enhanced Feature Fusion for Thermal-color Semantic Segmentation](https://mlsanity.com/paper/2204.10266/) (2022)
- [When SAM Meets Sonar Images](https://mlsanity.com/paper/2306.14109/) (2023)
- [Segment Any Building](https://mlsanity.com/paper/2310.01164/) (2023)
- [Unsupervised Image-generation Enhanced Adaptation for Object Detection in Thermal images](https://mlsanity.com/paper/2002.06770/) (2020)
- [Enhancing Object Detection in Adverse Conditions using Thermal Imaging](https://mlsanity.com/paper/1909.13551/) (2019)
- [Small and Dim Target Detection in IR Imagery: A Review](https://mlsanity.com/paper/2311.16346/) (2023)
- [ARTSeg: Employing Attention for Thermal images Semantic Segmentation](https://mlsanity.com/paper/2111.15257/) (2021)

## Built on by (most cited first, 8 of 8)

- [There is no SAMantics! Exploring SAM as a Backbone for Visual Understanding Tasks](https://mlsanity.com/paper/2411.15288/) (2024)
- [Semantics from Space: Satellite-Guided Thermal Semantic Segmentation Annotation for Aerial Field Robots](https://mlsanity.com/paper/2403.14056/) (2024)
- [AMLID: An Adaptive Multispectral Landmine Identification Dataset for Drone-Based Detection](https://mlsanity.com/paper/2512.18738/) (2025)
- [SAM2-ELNet: Label Enhancement and Automatic Annotation for Remote Sensing Segmentation](https://mlsanity.com/paper/2503.12404/) (2025)
- [Evaluating an Adaptive Multispectral Turret System for Autonomous Tracking Across Variable Illumination Conditions](https://mlsanity.com/paper/2512.22263/) (2025)
- [VORTEX: A Spatial Computing Framework for Optimized Drone Telemetry Extraction from First-Person View Flight Data](https://mlsanity.com/paper/2412.18505/) (2024)
- [Transferability of labels between multilens cameras](https://mlsanity.com/paper/2501.11513/) (2025)
- [Multi-temporal Adaptive Red-Green-Blue and Long-Wave Infrared Fusion for You Only Look Once-Based Landmine Detection from Unmanned Aerial Systems](https://mlsanity.com/paper/2512.20487/) (2025)

## Abstract (arXiv)

Segment Anything Model (SAM) is drastically accelerating the speed and accuracy of automatically segmenting and labeling large Red-Green-Blue (RGB) imagery datasets. However, SAM is unable to segment and label images outside of the visible light spectrum, for example, for multispectral or hyperspectral imagery. Therefore, this paper outlines a method we call the Multispectral Automated Transfer Technique (MATT). By transposing SAM segmentation masks from RGB images we can automatically segment and label multispectral imagery with high precision and efficiency. For example, the results demonstrate that segmenting and labeling a 2,400-image dataset utilizing MATT achieves a time reduction of 87.8% in developing a trained model, reducing roughly 20 hours of manual labeling, to only 2.4 hours. This efficiency gain is associated with only a 6.7% decrease in overall mean average precision (mAP) when training multispectral models via MATT, compared to a manually labeled dataset. We consider this an acceptable level of precision loss when considering the time saved during training, especially for rapidly prototyping experimental modeling methods. This research greatly contributes to the study of multispectral object detection by providing a novel and open-source method to rapidly segment, label, and train multispectral object detection models with minimal human interaction. Future research needs to focus on applying these methods to (i) space-based multispectral, and (ii) drone-based hyperspectral imagery.