# An Integrated Vision-and-Language Pretraining (VLP) and Visual Question Answering (VQA) model to Automate Nondestructive Evaluation Image Analysis

arXiv: https://arxiv.org/abs/2608.29408 · HTML: https://mlsanity.com/paper/2608.29408/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 6 of its 22 references that are in the map.
- First posted on arXiv 29 Aug 2026.

## Builds on (most cited first, 6 of 6)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [On Layer Normalization in the Transformer Architecture](https://mlsanity.com/paper/2002.04745/) (2020)
- [Vision-Language Pre-training: Basics, Recent Advances, and Future Trends](https://mlsanity.com/paper/2210.09263/) (2022)
- [Efficient Methods for Natural Language Processing: A Survey](https://mlsanity.com/paper/2209.00099/) (2022)
- [Visual Question Answering: Datasets, Algorithms, and Future Challenges](https://mlsanity.com/paper/1610.01465/) (2016)

## Abstract (arXiv)

An AI-based approach called ChatNDE Figure to Caption is introduced, which aims to automate the interpretation of NDE images using deep learning and natural language processing (NLP). A Vision-and-Language Pretraining (VLP) strategy is developed to help the model learn how to connect visual features with meaningful language. Basically, we built a large NDE image dataset, trained the model using annotated examples, and then evaluated how well it performed using BLEU scores to compare its output to expert written descriptions. So, the system combines a ResNet50 model to extract important features from the images and a GPT2 language model to turn those features into natural sounding text. Even though the accuracy of the model has been low the generated caption results have been solid so far, the captions were shorter but mentioned some important features of images what human experts would say, which shows the model is learning to pick up on key details. Also, a Visual Question Answering (VQA) model is used as part of the system. VQA models are designed to take an image and a question about that image (like Is there a crack? or Where is the defect located?) and generate a useful answer. By adding this layer, the platform will not just describe what it sees, it can also respond to specific questions, making it even more interactive and helpful for inspectors in the field. This whole approach is a big step toward speeding up NDE workflows, reducing human error, and making the technology more accessible.