mlsanityData to 7 Oct 2026

HEGEL: Hypergraph Transformer for Long Document Summarization

arXiv 2210.04126 ↗First posted 9 Oct 2022cs.CLcited by 28

Not on the map. Why?

Abstract (arXiv)

Extractive summarization for long documents is challenging due to the extended structured input context. The long-distance sentence dependency hinders cross-sentence relations modeling, the critical step of extractive summarization. This paper proposes HEGEL, a hypergraph neural network for long document summarization by capturing high-order cross-sentence relations. HEGEL updates and learns effective sentence representations with hypergraph transformer layers and fuses different types of sentence dependencies, including latent topics, keywords coreference, and section structure. We validate HEGEL by conducting extensive experiments on two benchmark datasets, and experimental results demonstrate the effectiveness and efficiency of HEGEL.

Key facts. Cited by 28 papers in the map (data through 7 Oct 2026). Builds on 28 of its 44 references that are in the map. First posted on arXiv 9 Oct 2022.

Builds on

28 of 44 references.

References · 28

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  4. 2017Graph Attention Networkscited by 8,854
  5. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  6. 2020Longformer: The Long-Document Transformercited by 3,362
  7. 2017Get To The Point: Summarization with Pointer-Generator Networkscited by 1,281
  8. 2015Teaching Machines to Read and Comprehendcited by 1,039
  9. 2016Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyondcited by 809
  10. 2019Text Summarization with Pretrained Encoderscited by 553
  11. 2018A Discourse-Aware Attention Model for Abstractive Summarization of Long Documentscited by 477
  12. 2016SummaRuNNer: A Recurrent Neural Network based Sequence Model for Extractive Summarization of Documentscited by 245
16 more
  1. 2020Graph-Bert: Only Attention is Needed for Learning Graph Representationscited by 176
  2. 2018Ranking Sentences for Extractive Summarization with Reinforcement Learningcited by 128
  3. 2016Neural Summarization by Extracting Sentences and Wordscited by 114
  4. 2020Heterogeneous Graph Neural Networks for Extractive Document Summarizationcited by 90
  5. 2019On Extractive and Abstractive Neural Document Summarization with Transformer Language Modelscited by 83
  6. 2020Be More with Less: Hypergraph Attention Networks for Inductive Text Classificationcited by 73
  7. 2019Sentence Centrality Revisited for Unsupervised Summarizationcited by 64
  8. 2019Extractive Summarization of Long Documents by Combining Global and Local Contextcited by 59
  9. 2017Graph-based Neural Multi-Document Summarizationcited by 54
  10. 2022HiStruct+: Improving Extractive Text Summarization with Hierarchical Structure Informationcited by 26
  11. 2020Enhancing Extractive Text Summarization with Topic-Aware Graph Neural Networkscited by 22
  12. 2020Systematically Exploring Redundancy Reduction in Summarizing Long Documentscited by 21
  13. 2022Improving the Faithfulness of Abstractive Summarization via Entity Coverage Controlcited by 21
  14. 2021Multiplex Graph Neural Network for Extractive Text Summarizationcited by 19
  15. 2021Topic-Guided Abstractive Multi-Document Summarizationcited by 19
  16. 2021Leveraging Information Bottleneck for Scientific Document Summarizationcited by 13

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 14; 2024: 9; 2025: 2; 2026: 1.

28 papers.

Sort:

By area:

  1. 2023Advancing Transformer Architecture in Long-Context Large Language Models: A Comprehensive Surveycited by 71
  2. 2023Extractive Summarization via ChatGPT for Faithful Summary Generationcited by 58
  3. 2023SummIt: Iterative Text Summarization via ChatGPTcited by 53
  4. 2024A Systematic Survey of Text Summarization: From Statistical Methods to Large Language Modelscited by 52
  5. 2023A Survey on Long Text Modeling with Transformerscited by 47
  6. 2023DiffuSum: Generation Enhanced Extractive Summarization with Diffusioncited by 25
  7. 2023Contrastive Hierarchical Discourse Graph for Scientific Document Summarizationcited by 9
  8. 2022Computing and Exploiting Document Structure to Improve Unsupervised Extractive Summarization of Legal Case Decisionscited by 8
  9. 2023Neural Natural Language Processing for Long Texts: A Survey on Classification and Summarizationcited by 7
  10. 2025StrucSum: Graph-Structured Reasoning for Long Document Extractive Summarization with LLMscited by 7
  11. 2024HDT: Hierarchical Document Transformercited by 6
  12. 2022Inverse Reinforcement Learning for Text Summarizationcited by 5
16 more
  1. 2023Hybrid Long Document Summarization using C2F-FAR and ChatGPT: A Practical Studycited by 5
  2. 2023Core Building Blocks: Next Gen Geo Spatial GPT Applicationcited by 4
  3. 2024Hierarchical Attention Graph for Scientific Document Summarization in Global and Local Levelcited by 4
  4. 2024DomainSum: A Hierarchical Benchmark for Fine-Grained Domain Shift in Abstractive Text Summarizationcited by 4
  5. 2024GraphLSS: Integrating Lexical, Structural, and Semantic Features for Long Document Extractive Summarizationcited by 4
  6. 2024PecSched: Preemptive and Efficient Cluster Scheduling for LLM Inferencecited by 3
  7. 2026Kan Extension Transformers: A Categorical Unification of Attention, Diffusion, and Predict-Detach Self-Conditioningcited by 3
  8. 2023Clinical Note Owns its Hierarchy: Multi-Level Hypergraph Neural Networks for Patient-Level Representation Learningcited by 2
  9. 2023Unsupervised Multi-document Summarization with Holistic Inferencecited by 2
  10. 2023Topology-guided Hypergraph Transformer Network: Unveiling Structural Insights for Improved Representationcited by 2
  11. 2023Learning Co-Speech Gesture for Multimodal Aphasia Type Detectioncited by 2
  12. 2024NextLevelBERT: Masked Language Modeling with Higher-Level Representations for Long Documentscited by 2
  13. 2025ResearchPulse: Building Method-Experiment Chains through Multi-Document Scientific Inferencecited by 2
  14. 2023Prototype-Enhanced Hypergraph Learning for Heterogeneous Information Networkscited by 1
  15. 2024T3: A Novel Zero-shot Transfer Learning Framework Iteratively Training on an Assistant Task for a Target Taskcited by 1
  16. 2024StructFormer: Document Structure-based Masked Attention and its Impact on Language Model Pre-Trainingcited by 0