mlsanityData to 14 Oct 2026

Attention Mechanism and Context Modeling System for Text Mining Machine Translation

arXiv 2408.04216 ↗First posted 8 Aug 2024cs.CL cs.AIcited by 15

Not on the map. Why?

Abstract (arXiv)

This paper advances a novel architectural schema anchored upon the Transformer paradigm and innovatively amalgamates the K-means categorization algorithm to augment the contextual apprehension capabilities of the schema. The transformer model performs well in machine translation tasks due to its parallel computing power and multi-head attention mechanism. However, it may encounter contextual ambiguity or ignore local features when dealing with highly complex language structures. To circumvent this constraint, this exposition incorporates the K-Means algorithm, which is used to stratify the lexis and idioms of the input textual matter, thereby facilitating superior identification and preservation of the local structure and contextual intelligence of the language. The advantage of this combination is that K-Means can automatically discover the topic or concept regions in the text, which may be directly related to translation quality. Consequently, the schema contrived herein enlists K-Means as a preparatory phase antecedent to the Transformer and recalibrates the multi-head attention weights to assist in the discrimination of lexis and idioms bearing analogous semantics or functionalities. This ensures the schema accords heightened regard to the contextual intelligence embodied by these clusters during the training phase, rather than merely focusing on locational intelligence.

Key facts. Cited by 15 papers in the map (data through 14 Oct 2026). Builds on 22 of its 36 references that are in the map. First posted on arXiv 8 Aug 2024.

Builds on

22 of 36 references.

References · 22

  1. 2023NDC-Scene: Boost Monocular 3D Semantic Scene Completion in Normalized Device Coordinates Spacecited by 90
  2. 2024Survival Prediction Across Diverse Cancer Types Using Neural Networkscited by 80
  3. 2023Deception Detection from Linguistic and Physiological Data Streams Using Bimodal Convolutional Neural Networkscited by 56
  4. 2023AttentionViz: A Global View of Transformer Attentioncited by 50
  5. 2024Convolutional neural network classification of cancer cytopathology images: taking breast cancer as an examplecited by 50
  6. 2024Exploring Diverse Methods in Visual Question Answeringcited by 45
  7. 2024Time Series Modeling for Heart Rate Prediction: From ARIMA to Transformerscited by 35
  8. 2024Efficiency optimization of large-scale language models based on deep learning in natural language processing taskscited by 14
  9. 2024Application of Multimodal Fusion Deep Learning Model in Disease Recognitioncited by 12
  10. 2024Advanced Financial Fraud Detection Using GNN-CL Modelcited by 11
  11. 2024An Enhanced Encoder-Decoder Network Architecture for Reducing Information Loss in Image Semantic Segmentationcited by 10
  12. 2024Exploration of Multi-Scale Image Fusion Systems in Intelligent Medical Image Analysiscited by 10
10 more
  1. 2024Integrating Medical Imaging and Clinical Reports Using Multimodal Deep Learning for Advanced Disease Analysiscited by 8
  2. 2024Research on Deep Learning Model of Feature Extraction Based on Convolutional Neural Networkcited by 7
  3. 2024Application of Computer Deep Learning Model in Diagnosis of Pulmonary Nodulescited by 7
  4. 2024Advancements in Feature Extraction Recognition of Medical Imaging Systems Through Deep Learning Techniquecited by 7
  5. 2024Research on Early Warning Model of Cardiovascular Disease Based on Computer Deep Learningcited by 6
  6. 2024Investigation of Customized Medical Decision Algorithms Utilizing Graph Neural Networkscited by 5
  7. 2024Advancing Financial Risk Prediction Through Optimized LSTM Model Performance and Comparative Analysiscited by 5
  8. 2024Research on Optimization of Natural Language Processing Model Based on Multimodal Deep Learningcited by 5
  9. 2024Advanced Multimodal Deep Learning Architecture for Image-Text Matchingcited by 4
  10. 2015Enhanced Bilingual Evaluation Understudycited by 3

Built on by

Cited by year

Citing papers per year: 2024: 13; 2025: 2.

15 papers.

Sort:

  1. 2024Self-Supervised Graph Neural Networks for Enhanced Feature Extraction in Heterogeneous Information Networkscited by 33
  2. 2024Optimizing Automated Picking Systems in Warehouse Robots Using Machine Learningcited by 20
  3. 2024YOLO-PPA based Efficient Traffic Sign Detection for Cruise Control in Autonomous Drivingcited by 16
  4. 2024Adversarial Neural Networks in Medical Imaging Advancements and Challenges in Semantic Segmentationcited by 10
  5. 2024Enhanced Recommendation Combining Collaborative Filtering and Large Language Modelscited by 10
  6. 2024Unveiling the Potential of Graph Neural Networks in SME Credit Risk Assessmentcited by 9
  7. 2024Balancing Innovation and Privacy: Data Security Strategies in Natural Language Processing Applicationscited by 8
  8. 2025User Behavior Analysis in Privacy Protection with Large Language Models: A Study on Privacy Preferences with Limited Datacited by 8
  9. 2024Enhancing Deep Learning with Optimized Gradient Descent: Bridging Numerical Methods and Neural Network Trainingcited by 7
  10. 2024Reducing Bias in Deep Learning Optimization: The RSGDM Approachcited by 7
  11. 2025LLM-Driven E-Commerce Marketing Content Optimization: Balancing Creativity and Conversioncited by 7
  12. 2024Root Cause Attribution of Delivery Risks via Causal Discovery with Reinforcement Learningcited by 5
3 more
  1. 2024Analysis and Design of a Personalized Recommendation System Based on a Dynamic User Interest Modelcited by 4
  2. 2024Enhanced Credit Score Prediction Using Ensemble Deep Learning Modelcited by 2
  3. 2024Enhancing Exchange Rate Forecasting with Explainable Deep Learning Modelscited by 2