mlsanityData to 14 Oct 2026

Exploration of Attention Mechanism-Enhanced Deep Learning Models in the Mining of Medical Textual Data

arXiv 2406.00016 ↗First posted 23 May 2024cs.CLcited by 11

Not on the map. Why?

Abstract (arXiv)

The research explores the utilization of a deep learning model employing an attention mechanism in medical text mining. It targets the challenge of analyzing unstructured text information within medical data. This research seeks to enhance the model's capability to identify essential medical information by incorporating deep learning and attention mechanisms. This paper reviews the basic principles and typical model architecture of attention mechanisms and shows the effectiveness of their application in the tasks of disease prediction, drug side effect monitoring, and entity relationship extraction. Aiming at the particularity of medical texts, an adaptive attention model integrating domain knowledge is proposed, and its ability to understand medical terms and process complex contexts is optimized. The experiment verifies the model's effectiveness in improving task accuracy and robustness, especially when dealing with long text. The future research path of enhancing model interpretation, realizing cross-domain knowledge transfer, and adapting to low-resource scenarios is discussed in the research outlook, which provides a new perspective and method support for intelligent medical information processing and clinical decision assistance. Finally, cross-domain knowledge transfer and adaptation strategies for low-resource scenarios, providing theoretical basis and technical reference for promoting the development of intelligent medical information processing and clinical decision support systems.

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Builds on 21 of its 37 references that are in the map. First posted on arXiv 23 May 2024.

Builds on

21 of 37 references.

References · 21

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,811
  3. 2019BioBERT: a pre-trained biomedical language representation model for biomedical text miningcited by 1,947
  4. 2020AdapterHub: A Framework for Adapting Transformerscited by 444
  5. 2018Fundamentals of Recurrent Neural Network (RNN) and Long Short-Term Memory (LSTM) Networkcited by 420
  6. 2018Traffic Graph Convolutional Recurrent Neural Network: A Deep Learning Framework for Network-Scale Traffic Learning and Forecastingcited by 146
  7. 2018Attention Branch Network: Learning of Attention Mechanism for Visual Explanationcited by 109
  8. 2019BERTje: A Dutch BERT Modelcited by 109
  9. 2024Survival Prediction Across Diverse Cancer Types Using Neural Networkscited by 80
  10. 2023Deception Detection from Linguistic and Physiological Data Streams Using Bimodal Convolutional Neural Networkscited by 56
  11. 2024Convolutional neural network classification of cancer cytopathology images: taking breast cancer as an examplecited by 50
  12. 2024Exploring Diverse Methods in Visual Question Answeringcited by 45
9 more
  1. 2023Improving Depth Gradient Continuity in Transformers: A Comparative Study on Monocular Depth Estimation with CNNcited by 35
  2. 2020DDANet: Dual Decoder Attention Network for Automatic Polyp Segmentationcited by 31
  3. 2024Regional Style and Color Transfercited by 30
  4. 20233D-Aware Neural Body Fitting for Occlusion Robust 3D Human Pose Estimationcited by 28
  5. 2021An Optimization-Based Meta-Learning Model for MRI Reconstruction with Diverse Datasetcited by 18
  6. 2019Constrained Attractor Selection Using Deep Reinforcement Learningcited by 17
  7. 2024Cross-Task Multi-Branch Vision Transformer for Facial Expression and Mask Wearing Classificationcited by 16
  8. 2020Attractor Selection in Nonlinear Energy Harvesting Using Deep Reinforcement Learningcited by 15
  9. 2015From Data Fusion to Knowledge Fusioncited by 10

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 9; 2025: 1; 2026: 1.

11 papers.

  1. 2024Adaptive Friction in Deep Learning: Enhancing Optimizers with Sigmoid and Tanh Functioncited by 16
  2. 2024A comparative study of generative adversarial networks for image recognition algorithms based on deep learning and traditional methodscited by 12
  3. 2024Advanced Financial Fraud Detection Using GNN-CL Modelcited by 11
  4. 2024Algorithm Research of ELMo Word Embedding and Deep Learning Multimodal Transformer in Image Descriptioncited by 10
  5. 2024Applying Conditional Generative Adversarial Networks for Imaging Diagnosiscited by 9
  6. 2024A Recommendation Model Utilizing Separation Embedding and Self-Attention for Feature Miningcited by 8
  7. 2024Exploiting Diffusion Prior for Out-of-Distribution Detectioncited by 7
  8. 2024Convolutional Neural Networks for Predictive Modeling of Lung Diseasecited by 7
  9. 2024Analyzing Diversity in Healthcare LLM Research: A Scientometric Perspectivecited by 4
  10. 2025Optimized detection of cyber-attacks on IoT networks via hybrid deep learning modelscited by 1
  11. 2026Attention-Based Deep Learning for Early Parkinson's Disease Detection with Tabular Biomedical Datacited by 0