# Semantic Representation Learning of Scientific Literature based on Adaptive Feature and Graph Neural Network

arXiv: https://arxiv.org/abs/2311.00296 · HTML: https://mlsanity.com/paper/2311.00296/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 56 references that are in the map.
- First posted on arXiv 1 Nov 2023.

## Builds on (most cited first, 11 of 11)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [A Comprehensive Survey on Graph Neural Networks](https://mlsanity.com/paper/1901.00596/) (2019)
- [SciBERT: A Pretrained Language Model for Scientific Text](https://mlsanity.com/paper/1903.10676/) (2019)
- [Text and Code Embeddings by Contrastive Pre-Training](https://mlsanity.com/paper/2201.10005/) (2022)
- [Deep Learning Based Text Classification: A Comprehensive Review](https://mlsanity.com/paper/2004.03705/) (2020)
- [Gather-Excite: Exploiting Feature Context in Convolutional Neural Networks](https://mlsanity.com/paper/1810.12348/) (2018)
- [A Survey on The Expressive Power of Graph Neural Networks](https://mlsanity.com/paper/2003.04078/) (2020)
- [Filter-enhanced MLP is All You Need for Sequential Recommendation](https://mlsanity.com/paper/2202.13556/) (2022)
- [Scientific and Technological Information Oriented Semantics-adversarial and Media-adversarial Cross-media Retrieval](https://mlsanity.com/paper/2203.08615/) (2022)
- [T2-GNN: Graph Neural Networks for Graphs with Incomplete Features and Structure via Teacher-Student Distillation](https://mlsanity.com/paper/2212.12738/) (2022)
- [A Comparative Study for Unsupervised Network Representation Learning](https://mlsanity.com/paper/1903.07902/) (2019)

## Abstract (arXiv)

Because most scientific literature data are unlabeled, semantic representation learning based on unsupervised graphs has become crucial. To enrich scientific-literature features, this paper proposes a semantic representation learning method based on adaptive features and graph neural networks. By introducing adaptive feature processing, scientific-literature features are considered globally and locally. The graph attention mechanism weights and aggregates features of scientific documents connected by citation relations, so that correlations among different documents can be expressed more effectively. In addition, an unsupervised graph neural network semantic representation learning method is proposed. By comparing the mutual information between positive and negative local semantic representations of scientific literature and the global graph semantic representation in the latent space, the graph neural network captures local and global information and improves semantic representation learning. Experimental results show that the proposed method is competitive for scientific literature classification.