# Algorithm Research of ELMo Word Embedding and Deep Learning Multimodal Transformer in Image Description

arXiv: https://arxiv.org/abs/2408.06357 · HTML: https://mlsanity.com/paper/2408.06357/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 27 references that are in the map.
- First posted on arXiv 26 Jul 2024.

## Builds on (most cited first, 12 of 12)

- [A Review of Generalized Zero-Shot Learning Methods](https://mlsanity.com/paper/2011.08641/) (2020)
- [Survival Prediction Across Diverse Cancer Types Using Neural Networks](https://mlsanity.com/paper/2404.08713/) (2024)
- [Improving Depth Gradient Continuity in Transformers: A Comparative Study on Monocular Depth Estimation with CNN](https://mlsanity.com/paper/2308.08333/) (2023)
- [A Comprehensive Survey on Word Representation Models: From Classical to State-Of-The-Art Word Representation Language Models](https://mlsanity.com/paper/2010.15036/) (2020)
- [Top-down Tree Long Short-Term Memory Networks](https://mlsanity.com/paper/1511.00060/) (2015)
- [Research on Image Recognition Technology Based on Multimodal Deep Learning](https://mlsanity.com/paper/2405.03091/) (2024)
- [Exploration of Attention Mechanism-Enhanced Deep Learning Models in the Mining of Medical Textual Data](https://mlsanity.com/paper/2406.00016/) (2024)
- [Research on Intelligent Aided Diagnosis System of Medical Image Based on Computer Deep Learning](https://mlsanity.com/paper/2404.18419/) (2024)
- [Integrating Medical Imaging and Clinical Reports Using Multimodal Deep Learning for Advanced Disease Analysis](https://mlsanity.com/paper/2405.17459/) (2024)
- [Investigation of Customized Medical Decision Algorithms Utilizing Graph Neural Networks](https://mlsanity.com/paper/2405.17460/) (2024)
- [Advancing Financial Risk Prediction Through Optimized LSTM Model Performance and Comparative Analysis](https://mlsanity.com/paper/2405.20603/) (2024)
- [Fine-tuning deep CNN models on specific MS COCO categories](https://mlsanity.com/paper/1709.01476/) (2017)

## Built on by (most cited first, 10 of 10)

- [Advanced RAG Models with Graph Structures: Optimizing Complex Knowledge Reasoning and Text Generation](https://mlsanity.com/paper/2411.03572/) (2024)
- [Wasserstein Distance-Weighted Adversarial Network for Cross-Domain Credit Risk Assessment](https://mlsanity.com/paper/2409.18544/) (2024)
- [Contrastive Learning for Knowledge-Based Question Generation in Large Language Models](https://mlsanity.com/paper/2409.13994/) (2024)
- [Applying Hybrid Graph Neural Networks to Strengthen Credit Risk Analysis](https://mlsanity.com/paper/2410.04283/) (2024)
- [A Recommendation Model Utilizing Separation Embedding and Self-Attention for Feature Mining](https://mlsanity.com/paper/2410.15026/) (2024)
- [Deep Learning for Medical Text Processing: BERT Model Fine-Tuning and Comparative Study](https://mlsanity.com/paper/2410.20792/) (2024)
- [Optimizing News Text Classification with Bi-LSTM and Attention Mechanism for Efficient Data Processing](https://mlsanity.com/paper/2409.15576/) (2024)
- [Deep Learning-Based Channel Squeeze U-Structure for Lung Nodule Detection and Segmentation](https://mlsanity.com/paper/2409.13868/) (2024)
- [Graph Neural Network Framework for Sentiment Analysis Using Syntactic Feature](https://mlsanity.com/paper/2409.14000/) (2024)
- [Optimizing Retrieval-Augmented Generation with Elasticsearch for Enhanced Question-Answering Systems](https://mlsanity.com/paper/2410.14167/) (2024)

## Abstract (arXiv)

Zero sample learning is an effective method for data deficiency. The existing embedded zero sample learning methods only use the known classes to construct the embedded space, so there is an overfitting of the known classes in the testing process. This project uses category semantic similarity measures to classify multiple tags. This enables it to incorporate unknown classes that have the same meaning as currently known classes into the vector space when it is built. At the same time, most of the existing zero sample learning algorithms directly use the depth features of medical images as input, and the feature extraction process does not consider semantic information. This project intends to take ELMo-MCT as the main task and obtain multiple visual features related to the original image through self-attention mechanism. In this paper, a large number of experiments are carried out on three zero-shot learning reference datasets, and the best harmonic average accuracy is obtained compared with the most advanced algorithms.