# Graph-to-Text Generation with Dynamic Structure Pruning

arXiv: https://arxiv.org/abs/2209.07258 · HTML: https://mlsanity.com/paper/2209.07258/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 47 references that are in the map.
- First posted on arXiv 15 Sep 2022.

## Builds on (most cited first, 30 of 30)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Fast Graph Representation Learning with PyTorch Geometric](https://mlsanity.com/paper/1903.02428/) (2019)
- [Modeling Relational Data with Graph Convolutional Networks](https://mlsanity.com/paper/1703.06103/) (2017)
- [Self-Attention with Relative Position Representations](https://mlsanity.com/paper/1803.02155/) (2018)
- [A Diversity-Promoting Objective Function for Neural Conversation Models](https://mlsanity.com/paper/1510.03055/) (2015)
- [MAD-X: An Adapter-Based Framework for Multi-Task Cross-Lingual Transfer](https://mlsanity.com/paper/2005.00052/) (2020)
- [Challenges in Data-to-Document Generation](https://mlsanity.com/paper/1707.08052/) (2017)
- [Text Generation from Knowledge Graphs with Graph Transformers](https://mlsanity.com/paper/1904.02342/) (2019)
- [Handling Divergent Reference Texts when Evaluating Table-to-Text Generation](https://mlsanity.com/paper/1906.01081/) (2019)
- [Graph Transformer for Graph-to-Sequence Learning](https://mlsanity.com/paper/1911.07470/) (2019)
- [Leveraging Graph to Improve Abstractive Multi-Document Summarization](https://mlsanity.com/paper/2005.10043/) (2020)
- [Long and Diverse Text Generation with Planning-based Hierarchical Variational Model](https://mlsanity.com/paper/1908.06605/) (2019)
- [Graph-to-Sequence Learning using Gated Graph Neural Networks](https://mlsanity.com/paper/1806.09835/) (2018)
- [A Graph-to-Sequence Model for AMR-to-Text Generation](https://mlsanity.com/paper/1805.02473/) (2018)
- [A Survey on Text-to-SQL Parsing: Concepts, Methods, and Future Directions](https://mlsanity.com/paper/2208.13629/) (2022)
- [Structural Adapters in Pretrained Language Models for AMR-to-text Generation](https://mlsanity.com/paper/2103.09120/) (2021)
- [A Survey on Complex Question Answering over Knowledge Base: Recent Advances and Challenges](https://mlsanity.com/paper/2007.13069/) (2020)
- [S2SQL: Injecting Syntax to Question-Schema Interaction Graph Encoder for Text-to-SQL Parsers](https://mlsanity.com/paper/2203.06958/) (2022)
- [Modeling Graph Structure in Transformer for Better AMR-to-Text Generation](https://mlsanity.com/paper/1909.00136/) (2019)
- [Proton: Probing Schema Linking Information from Pre-trained Language Models for Text-to-SQL Parsing](https://mlsanity.com/paper/2206.14017/) (2022)
- [Densely Connected Graph Convolutional Networks for Graph-to-Sequence Learning](https://mlsanity.com/paper/1908.05957/) (2019)
- [Deep Graph Convolutional Encoders for Structured Data to Text Generation](https://mlsanity.com/paper/1810.09995/) (2018)
- [Towards a Decomposable Metric for Explainable Evaluation of Text Generation from AMR](https://mlsanity.com/paper/2008.08896/) (2020)
- [Structural Neural Encoders for AMR-to-text Generation](https://mlsanity.com/paper/1903.11410/) (2019)
- [Not All Attention Is Needed: Gated Attention Network for Sequence Data](https://mlsanity.com/paper/1912.00349/) (2019)
- [Lightweight, Dynamic Graph Convolutional Networks for AMR-to-Text Generation](https://mlsanity.com/paper/2010.04383/) (2020)

## Built on by (most cited first, 2 of 2)

- [Retrieval-Augmented Generation with Graphs (GraphRAG)](https://mlsanity.com/paper/2501.00309/) (2024)
- [Unifying Structured Data as Graph for Data-to-Text Pre-Training](https://mlsanity.com/paper/2401.01183/) (2024)

## Abstract (arXiv)

Most graph-to-text works are built on the encoder-decoder framework with cross-attention mechanism. Recent studies have shown that explicitly modeling the input graph structure can significantly improve the performance. However, the vanilla structural encoder cannot capture all specialized information in a single forward pass for all decoding steps, resulting in inaccurate semantic representations. Meanwhile, the input graph is flatted as an unordered sequence in the cross attention, ignoring the original graph structure. As a result, the obtained input graph context vector in the decoder may be flawed. To address these issues, we propose a Structure-Aware Cross-Attention (SACA) mechanism to re-encode the input graph representation conditioning on the newly generated context at each decoding step in a structure aware manner. We further adapt SACA and introduce its variant Dynamic Graph Pruning (DGP) mechanism to dynamically drop irrelevant nodes in the decoding process. We achieve new state-of-the-art results on two graph-to-text datasets, LDC2020T02 and ENT-DESC, with only minor increase on computational cost.