mlsanityData to 7 Oct 2026

Enhancing Keyphrase Extraction from Long Scientific Documents using Graph Embeddings

arXiv 2305.09316 ↗First posted 16 May 2023cs.CL cs.AIcited by 3

Not on the map. Why?

Abstract (arXiv)

In this study, we investigate using graph neural network (GNN) representations to enhance contextualized representations of pre-trained language models (PLMs) for keyphrase extraction from lengthy documents. We show that augmenting a PLM with graph embeddings provides a more comprehensive semantic understanding of words in a document, particularly for long documents. We construct a co-occurrence graph of the text and embed it using a graph convolutional network (GCN) trained on the task of edge prediction. We propose a graph-enhanced sequence tagging architecture that augments contextualized PLM embeddings with graph representations. Evaluating on benchmark datasets, we demonstrate that enhancing PLMs with graph embeddings outperforms state-of-the-art models on long documents, showing significant improvements in F1 scores across all the datasets. Our study highlights the potential of GNN representations as a complementary approach to improve PLM performance for keyphrase extraction from long documents.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 36 of its 87 references that are in the map. First posted on arXiv 16 May 2023.

Builds on

36 of 87 references.

References · 36

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  4. 2017Graph Attention Networkscited by 8,854
  5. 2017Inductive Representation Learning on Large Graphscited by 6,284
  6. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,346
  7. 2020Longformer: The Long-Document Transformercited by 3,362
  8. 2019BioBERT: a pre-trained biomedical language representation model for biomedical text miningcited by 1,935
  9. 2016node2vec: Scalable Feature Learning for Networkscited by 1,913
  10. 2020Big Bird: Transformers for Longer Sequencescited by 1,673
  11. 2014DeepWalk: Online Learning of Social Representationscited by 1,543
  12. 2019SciBERT: A Pretrained Language Model for Scientific Textcited by 1,404
24 more
  1. 2018Graph Convolutional Neural Networks for Web-Scale Recommender Systemscited by 1,133
  2. 2019Publicly Available Clinical BERT Embeddingscited by 995
  3. 2015Bidirectional LSTM-CRF Models for Sequence Taggingcited by 544
  4. 2020FinBERT: A Pretrained Language Model for Financial Communicationscited by 161
  5. 2020LEGAL-BERT: The Muppets straight out of Law Schoolcited by 111
  6. 2019ScisummNet: A Large Annotated Corpus and Content-Impact Models for Scientific Paper Summarization with Citation Networkscited by 73
  7. 2018Simple Unsupervised Keyphrase Extraction using Sentence Embeddingscited by 62
  8. 2021Learning Rich Representation of Keyphrases from Textcited by 36
  9. 2020Beyond 512 Tokens: Siamese Multi-depth Transformer-based Hierarchical Encoder for Long-Form Document Matchingcited by 35
  10. 2022Efficient Classification of Long Documents Using Transformerscited by 30
  11. 2020Self-Supervised Learning of Contextual Embeddings for Link Prediction in Heterogeneous Networkscited by 17
  12. 2019DivGraphPointer: A Graph Pointer Network for Extracting Diverse Keyphrasescited by 16
  13. 2020TNT-KID: Transformer-based Neural Tagger for Keyword Identificationcited by 15
  14. 2021Heterogeneous Graph Neural Networks for Keyphrase Generationcited by 15
  15. 2022LDKP: A Dataset for Identifying Keyphrases from Long Scientific Documentscited by 13
  16. 2021Keyphrase Generation Beyond the Boundaries of Title and Abstractcited by 12
  17. 2019Keyphrase Generation: A Multi-Aspect Surveycited by 8
  18. 2019S2ORC: The Semantic Scholar Open Research Corpuscited by 8
  19. 2022Understanding Long Documents with Different Position-Aware Attentionscited by 7
  20. 2021NodeSim: Node Similarity based Network Embedding for Diverse Link Predictioncited by 6
  21. 2018Theme-weighted Ranking of Keywords from Text Documents using Phrase Embeddingscited by 5
  22. 2021Phraseformer: Multimodal Key-phrase Extraction using Transformer and Graph Embeddingcited by 4
  23. 2022Link Prediction with Contextualized Self-Supervisioncited by 4
  24. 2022Query-Based Keyphrase Extraction from Long Documentscited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 0; 2026: 2.

3 papers.

  1. 2024LongKey: Keyphrase Extraction for Long Documentscited by 1
  2. 2026Attention Expansion: Enhancing Keyphrase Extraction from Long Documents with Attention-Augmented Contextualized Embeddingscited by 0
  3. 2026Semantic Networks as Clues: A Theoretical Foundation and Process Optimization for Semantic Network Constructioncited by 0