# Failure-Aware Refinement of Vision-Language Model for Lithography Defect Detection

arXiv: https://arxiv.org/abs/2606.08908 · HTML: https://mlsanity.com/paper/2606.08908/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Chart understanding and anomaly detection, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 5 of its 15 references that are in the map.
- First posted on arXiv 8 Jun 2026.

## Placed in

- Multimodal LLMs / vision-language models / [Chart understanding and anomaly detection](https://mlsanity.com/mllm/chart-understanding-and-anomaly-detection/)

## Builds on (most cited first, 5 of 5)

- [Qwen3-VL Technical Report](https://mlsanity.com/paper/2511.21631/) (2025)
- [VLM-R1: A Stable and Generalizable R1-style Large Vision-Language Model](https://mlsanity.com/paper/2504.07615/) (2025)
- [Effectiveness Assessment of Recent Large Vision-Language Models](https://mlsanity.com/paper/2403.04306/) (2024)
- [Deep Learning-Based Defect Classification and Detection in SEM Images](https://mlsanity.com/paper/2206.13505/) (2022)
- [A Physics-Constrained, Design-Driven Methodology for Defect Dataset Generation in Optical Lithography](https://mlsanity.com/paper/2512.09001/) (2025)

## Abstract (arXiv)

Semiconductor lithography inspection requires reliable detection of small pattern defects such as bridge, burr, pinch, and contamination. In this study, we propose a two-stage vision-language framework that combines initial defect detection with prediction refinement. In the first stage, Qwen3-VL is fine-tuned with LoRA as a vision-language adapter to predict defect counts, defect categories, and normalized bounding boxes from lithography images. However, direct fine-tuning may still produce common test-time errors, including false positives, missed defects, and incorrect defect types. To address this limitation, the second stage trains a refinement module using first-stage prediction failures and their corrected labels, allowing the model to review and revise initial outputs. By learning from cases where the initial adapter fails, the refinement process improves defect inference beyond single-stage fine-tuning.