A Novel Ehanced Move Recognition Algorithm Based on Pre-trained Models with Positional Embeddings
Not on the map. Why?
Abstract (arXiv)
The recognition of abstracts is crucial for effectively locating the content and clarifying the article. Existing move recognition algorithms lack the ability to learn word position information to obtain contextual semantics. This paper proposes a novel enhanced move recognition algorithm with an improved pre-trained model and a gated network with attention mechanism for unstructured abstracts of Chinese scientific and technological papers. The proposed algorithm first performs summary data segmentation and vocabulary training. The EP-ERNIE_AT-GRU framework is leveraged to incorporate word positional information, facilitating deep semantic learning and targeted feature extraction. Experimental results demonstrate that the proposed algorithm achieves 13.37% higher accuracy on the split dataset than on the original dataset and a 7.55% improvement in accuracy over the basic comparison model.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 15 of its 48 references that are in the map. First posted on arXiv 14 Aug 2023.
Builds on
15 of 48 references.
References · 15
- 2017Attention Is All You Needcited by 60,361
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
- 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,790
- 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,989
- 2019Transformer-XL: Attentive Language Models Beyond a Fixed-Length Contextcited by 1,850
- 2021Conditional Positional Encodings for Vision Transformerscited by 406
- 2019Text Classification Algorithms: A Surveycited by 163
- 2020Deep Learning Based Text Classification: A Comprehensive Reviewcited by 141
- 2019Attention in Natural Language Processingcited by 101
- 2022The Topological BERT: Transforming Attention into Topology for Natural Language Processingcited by 10
- 2023ERNIE 3.0 Tiny: Frustratingly Simple Method to Improve Task-Agnostic Distillation Generalizationcited by 3
3 more
- 2023SAT: Size-Aware Transformer for 3D Point Cloud Semantic Segmentationcited by 3
- 2023Dynamic Mobile-Former: Strengthening Dynamic Convolution with Attention and Residual Connection in Kernel Spacecited by 2
- 2021A Novel Deep Reinforcement Learning Based Stock Direction Prediction using Knowledge Graph and Community Aware Sentimentscited by 1
Built on by
Not cited yet.