mlsanityData to 7 Oct 2026

FabGPT: An Efficient Large Multimodal Model for Complex Wafer Defect Knowledge Queries

arXiv 2407.10810 ↗First posted 15 Jul 2024cs.CV cs.AI cs.ARcited by 12

Not on the map. Why?

Abstract (arXiv)

Intelligence is key to advancing integrated circuit (IC) fabrication. Recent breakthroughs in Large Multimodal Models (LMMs) have unlocked extraditionary abilities in understanding images and text, fostering intelligent fabrication. Leveraging the power of LMMs, we introduce FabGPT, a customized IC fabrication large multimodal model for wafer defect knowledge query. FabGPT manifests expertise in conducting defect detection in Scanning Electron Microscope (SEM) images, performing root cause analysis, and providing expert Q&A on fabrication processes. FabGPT matches enhanced multimodal features to automatically detect minute defects under complex wafer backgrounds and reduce the subjectivity of manual threshold settings. Besides, the proposed modulation module and interactive corpus training strategy embed wafer defect knowledge into the pre-trained model, effectively balancing Q&A queries related to defect knowledge and original knowledge and mitigating the modality bias issues. Experiments on in-house fab data show that FabGPT achieves significant performance improvement in wafer defect detection and knowledge querying.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 20 of its 33 references that are in the map. First posted on arXiv 15 Jul 2024.

Builds on

20 of 33 references.

References · 20

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  5. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  6. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,689
  7. 2023ImageBind: One Embedding Space To Bind Them Allcited by 1,162
  8. 2021Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNetcited by 1,096
  9. 2023LISA: Reasoning Segmentation via Large Language Modelcited by 770
  10. 2018CCNet: Criss-Cross Attention for Semantic Segmentationcited by 717
  11. 2017Learning multiple visual domains with residual adapterscited by 553
  12. 2023AnomalyGPT: Detecting Industrial Anomalies Using Large Vision-Language Modelscited by 169
8 more
  1. 2022PP-OCRv3: More Attempts for the Improvement of Ultra Lightweight OCR Systemcited by 105
  2. 2022Catching Both Gray and Black Swans: Open-set Supervised Anomaly Detectioncited by 104
  3. 2022Explicit Boundary Guided Semi-Push-Pull Contrastive Learning for Supervised Anomaly Detectioncited by 72
  4. 2023Symbol tuning improves in-context learning in language modelscited by 72
  5. 2021Explainable Deep Few-shot Anomaly Detection with Deviation Networkscited by 69
  6. 2022Prototypical Residual Networks for Anomaly Detection and Localizationcited by 56
  7. 2023Pre-Training to Learn in Contextcited by 41
  8. 2020Bidirectional Encoder Representations from Transformers (BERT): A sentiment analysis odysseycited by 21

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 5; 2026: 6.

12 papers.

  1. 2024MMAD: A Comprehensive Benchmark for Multimodal Large Language Models in Industrial Anomaly Detectioncited by 46
  2. 2025FiLo++: Zero-/Few-Shot Anomaly Detection by Fused Fine-Grained Descriptions and Deformable Localizationcited by 7
  3. 2025Multimodal RAG-driven Anomaly Detection and Classification in Laser Powder Bed Fusion using Large Language Modelscited by 3
  4. 2026EAGLE: Expert-Augmented Attention Guidance for Tuning-Free Industrial Anomaly Detection in Multimodal Large Language Modelscited by 2
  5. 2025GenEDA: Towards Generative Netlist Functional Reasoning via Cross-Modal Circuit Encoder-Decoder Alignmentcited by 1
  6. 2026UniPCB: A Unified Vision-Language Benchmark for Open-Ended PCB Quality Inspectioncited by 1
  7. 2026MMR-AD: A Large-Scale Multimodal Dataset for Benchmarking General Anomaly Detection with Multimodal Large Language Modelscited by 1
  8. 2025Unsupervised Integrated-Circuit Defect Segmentation via Image-Intrinsic Normalitycited by 0
  9. 2025Image-Intrinsic Priors for Integrated Circuit Defect Detection and Novel Class Discovery via Self-Supervised Learningcited by 0
  10. 2026CiQi-Agent: Aligning Vision, Tools and Aesthetics in Multimodal Agent for Cultural Reasoning on Chinese Porcelainscited by 0
  11. 2026A Hybrid Vision-Language Architecture for Automated Defect Reasoning and Report Generation in Industrial Inspectioncited by 0
  12. 2026LithoDreamer: A Physics-Informed World Model for Multi-Stage Computational Lithographycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Towards Open-Vocabulary Industrial Defect Understanding with a Large-Scale Multimodal Dataset
  2. 2026WaferSAGE: Large Language Model-Powered Wafer Defect Analysis via Synthetic Data Generation and Rubric-Guided Reinforcement Learning
  3. 2026LDU-Bench: Multimodal LLM Evaluation for Lithography Defect Understanding under Layout-Varying Circuit Backgrounds
  4. 2026Synthetic Defect Image Generation for Power Line Insulator Inspection Using Multimodal Large Language Models
  5. 2024Multi-Modal Instruction-Tuning Small-Scale Language-and-Vision Assistant for Semiconductor Electron Micrograph Analysis
  6. 2026Failure-Aware Refinement of Vision-Language Model for Lithography Defect Detection
  7. 2024Addressing Class Imbalance and Data Limitations in Advanced Node Semiconductor Defect Inspection: A Generative Approach for SEM Images
  8. 2026Efficient Patch-Based Anomaly Detection Fused with Diffusion Driven Generative Modeling for Semiconductor Wafer Bin Map Open Set Anomaly Detection