# Sanitizing Manufacturing Dataset Labels Using Vision-Language Models

arXiv: https://arxiv.org/abs/2506.23465 · HTML: https://mlsanity.com/paper/2506.23465/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Placed in Prompt learning and adaptation of CLIP, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 20 of its 46 references that are in the map.
- First posted on arXiv 30 Jun 2025.

## Placed in

- Multimodal LLMs / vision-language models / [Prompt learning and adaptation of CLIP](https://mlsanity.com/mllm/prompt-learning-and-adaptation-of-clip/)

## Builds on (most cited first, 20 of 20)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [LAION-5B: An open large-scale dataset for training next generation image-text models](https://mlsanity.com/paper/2210.08402/) (2022)
- [Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labels](https://mlsanity.com/paper/1804.06872/) (2018)
- [Symmetric Cross Entropy for Robust Learning with Noisy Labels](https://mlsanity.com/paper/1908.06112/) (2019)
- [Early-Learning Regularization Prevents Memorization of Noisy Labels](https://mlsanity.com/paper/2007.00151/) (2020)
- [Webly Supervised Learning of Convolutional Networks](https://mlsanity.com/paper/1505.01554/) (2015)
- [Sample Selection with Uncertainty of Losses for Learning with Noisy Labels](https://mlsanity.com/paper/2106.00445/) (2021)
- [Learning From Noisy Singly-labeled Data](https://mlsanity.com/paper/1712.04577/) (2017)
- [Active label cleaning for improved dataset quality under resource constraints](https://mlsanity.com/paper/2109.00574/) (2021)
- [A Comparative Study on Annotation Quality of Crowdsourcing and LLM via Label Aggregation](https://mlsanity.com/paper/2401.09760/) (2024)
- [CLIPCleaner: Cleaning Noisy Labels with CLIP](https://mlsanity.com/paper/2408.10012/) (2024)
- [Vision-Language Models are Strong Noisy Label Detectors](https://mlsanity.com/paper/2409.19696/) (2024)
- [A Systematic Evaluation of LLM Strategies for Mental Health Text Analysis: Fine-tuning vs. Prompt Engineering vs. RAG](https://mlsanity.com/paper/2503.24307/) (2025)
- [Towards Automatic Construction of Diverse, High-quality Image Dataset](https://mlsanity.com/paper/1708.06495/) (2017)
- [Few-Shot Adversarial Low-Rank Fine-Tuning of Vision-Language Models](https://mlsanity.com/paper/2505.15130/) (2025)
- [A Survey on Data Cleaning Methods for Improved Machine Learning Model Performance](https://mlsanity.com/paper/2109.07127/) (2021)
- [Multi-Modal Multi-Task (M3T) Federated Foundation Models for Embodied AI: Potentials and Challenges for Edge Integration](https://mlsanity.com/paper/2505.11191/) (2025)
- [Unsupervised Time-Series Signal Analysis with Autoencoders and Vision Transformers: A Review of Architectures and Applications](https://mlsanity.com/paper/2504.16972/) (2025)
- [Enhancing Glass Defect Detection with Diffusion Models: Addressing Imbalanced Datasets in Manufacturing Quality Control](https://mlsanity.com/paper/2505.03134/) (2025)
- [GRU-AUNet: A Domain Adaptation Framework for Contactless Fingerprint Presentation Attack Detection](https://mlsanity.com/paper/2504.01213/) (2025)

## Built on by (most cited first, 6 of 6)

- [pFedMMA: Personalized Federated Fine-Tuning with Multi-Modal Adapter for Vision-Language Models](https://mlsanity.com/paper/2507.05394/) (2025)
- [GPS Spoofing Attack Detection in Autonomous Vehicles Using Adaptive DBSCAN](https://mlsanity.com/paper/2510.10766/) (2025)
- [Ensembling Multilingual Transformers for Robust Sentiment Analysis of Tweets](https://mlsanity.com/paper/2509.24080/) (2025)
- [Adaptive Edge-Cloud Inference for Speech-to-Action Systems Using ASR and Large Language Models](https://mlsanity.com/paper/2512.12769/) (2025)
- [Same Game, Different Story: A Minimal Conservative Strategic Robustness Benchmark for Large Language Model Agents](https://mlsanity.com/paper/2607.19670/) (2026)
- [Enhancing Automatic Modulation Recognition With a Reconstruction-Driven Vision Transformer Under Limited Labels](https://mlsanity.com/paper/2508.20193/) (2025)

## Abstract (arXiv)

The success of machine learning models in industrial applications is heavily dependent on the quality of the datasets used to train the models. However, large-scale datasets, specially those constructed from crowd-sourcing and web-scraping, often suffer from label noise, inconsistencies, and errors. This problem is particularly pronounced in manufacturing domains, where obtaining high-quality labels is costly and time-consuming. This paper introduces Vision-Language Sanitization and Refinement (VLSR), which is a vision-language-based framework for label sanitization and refinement in multi-label manufacturing image datasets. This method embeds both images and their associated textual labels into a shared semantic space leveraging the CLIP vision-language model. Then two key tasks are addressed in this process by computing the cosine similarity between embeddings. First, label sanitization is performed to identify irrelevant, misspelled, or semantically weak labels, and surface the most semantically aligned label for each image by comparing image-label pairs using cosine similarity between image and label embeddings. Second, the method applies density-based clustering on text embeddings, followed by iterative cluster merging, to group semantically similar labels into unified label groups. The Factorynet dataset, which includes noisy labels from both human annotations and web-scraped sources, is employed to evaluate the effectiveness of the proposed framework. Experimental results demonstrate that the VLSR framework successfully identifies problematic labels and improves label consistency. This method enables a significant reduction in label vocabulary through clustering, which ultimately enhances the dataset's quality for training robust machine learning models in industrial applications with minimal human intervention.