# Integrating Vision and Location with Transformers: A Multimodal Deep Learning Framework for Medical Wound Analysis

arXiv: https://arxiv.org/abs/2504.10452 · HTML: https://mlsanity.com/paper/2504.10452/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 35 references that are in the map.
- First posted on arXiv 14 Apr 2025.

## Builds on (most cited first, 10 of 10)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [An Empirical Study of Training Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.02057/) (2021)
- [A Survey of Transformers](https://mlsanity.com/paper/2106.04554/) (2021)
- [Modeling Recurrence for Transformer](https://mlsanity.com/paper/1904.03092/) (2019)
- [Detect-and-Segment: a Deep Learning Approach to Automate Wound Image Segmentation](https://mlsanity.com/paper/2111.01590/) (2021)
- [Multi-modal Wound Classification using Wound Image and Location by Deep Neural Network](https://mlsanity.com/paper/2109.06969/) (2021)
- [Integrated Image and Location Analysis for Wound Classification: A Deep Learning Approach](https://mlsanity.com/paper/2308.11877/) (2023)
- [Vision Transformer for femur fracture classification](https://mlsanity.com/paper/2108.03414/) (2021)
- [Development of a deep learning-based tool to assist wound classification](https://mlsanity.com/paper/2303.16522/) (2023)

## Built on by (most cited first, 2 of 2)

- [Multi-modal wound classification using wound image and location by Xception and Gaussian Mixture Recurrent Neural Network (GMRNN)](https://mlsanity.com/paper/2505.08086/) (2025)
- [Robust Pre-Training of Medical Vision-and-Language Models with Domain-Invariant Multi-Modal Masked Reconstruction](https://mlsanity.com/paper/2602.17689/) (2026)

## Abstract (arXiv)

Effective recognition of acute and difficult-to-heal wounds is a necessary step in wound diagnosis. An efficient classification model can help wound specialists classify wound types with less financial and time costs and also help in deciding on the optimal treatment method. Traditional machine learning models suffer from feature selection and are usually cumbersome models for accurate recognition. Recently, deep learning (DL) has emerged as a powerful tool in wound diagnosis. Although DL seems promising for wound type recognition, there is still a large scope for improving the efficiency and accuracy of the model. In this study, a DL-based multimodal classifier was developed using wound images and their corresponding locations to classify them into multiple classes, including diabetic, pressure, surgical, and venous ulcers. A body map was also created to provide location data, which can help wound specialists label wound locations more effectively. The model uses a Vision Transformer to extract hierarchical features from input images, a Discrete Wavelet Transform (DWT) layer to capture low and high frequency components, and a Transformer to extract spatial features. The number of neurons and weight vector optimization were performed using three swarm-based optimization techniques (Monster Gorilla Toner (MGTO), Improved Gray Wolf Optimization (IGWO), and Fox Optimization Algorithm). The evaluation results show that weight vector optimization using optimization algorithms can increase diagnostic accuracy and make it a very effective approach for wound detection. In the classification using the original body map, the proposed model was able to achieve an accuracy of 0.8123 using image data and an accuracy of 0.8007 using a combination of image data and wound location. Also, the accuracy of the model in combination with the optimization models varied from 0.7801 to 0.8342.