mlsanityData to 7 Oct 2026

Structural Adapters in Pretrained Language Models for AMR-to-text Generation

arXiv 2103.09120 ↗First posted 16 Mar 2021cs.CLcited by 48

Not on the map. Why?

Abstract (arXiv)

Pretrained language models (PLM) have recently advanced graph-to-text generation, where the input graph is linearized into a sequence and fed into the PLM to obtain its representation. However, efficiently encoding the graph structure in PLMs is challenging because such models were pretrained on natural language, and modeling structured data may lead to catastrophic forgetting of distributional knowledge. In this paper, we propose StructAdapt, an adapter method to encode graph structure into PLMs. Contrary to prior work, StructAdapt effectively models interactions among the nodes based on the graph connectivity, only training graph structure-aware adapter parameters. In this way, we incorporate task-specific knowledge while maintaining the topological structure of the graph. We empirically show the benefits of explicitly encoding graph structure into PLMs using StructAdapt, outperforming the state of the art on two AMR-to-text datasets, training only 5.1% of the PLM parameters.

Key facts. Cited by 48 papers in the map (data through 7 Oct 2026). Builds on 49 of its 65 references that are in the map. First posted on arXiv 16 Mar 2021.

Builds on

49 of 65 references.

References · 49

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  5. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  6. 2017Graph Attention Networkscited by 8,854
  7. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  8. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  9. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  10. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  11. 2018How Powerful are Graph Neural Networks?cited by 4,355
  12. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
37 more
  1. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  2. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
  3. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,201
  4. 2019SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systemscited by 1,718
  5. 2017Modeling Relational Data with Graph Convolutional Networkscited by 1,684
  6. 2020AdapterFusion: Non-Destructive Task Composition for Transfer Learningcited by 780
  7. 2013An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networkscited by 711
  8. 2017Learning multiple visual domains with residual adapterscited by 553
  9. 2020MAD-X: An Adapter-Based Framework for Multi-Task Cross-Lingual Transfercited by 443
  10. 2020AdapterHub: A Framework for Adapting Transformerscited by 441
  11. 2020ToTTo: A Controlled Table-To-Text Generation Datasetcited by 267
  12. 2021WARP: Word-level Adversarial ReProgrammingcited by 220
  13. 2020DART: Open-Domain Structured Data Record to Text Generationcited by 144
  14. 2020Investigating Pretrained Language Models for Graph-to-Text Generationcited by 117
  15. 2020Text-to-Text Pre-Training for Data-to-Text Taskscited by 104
  16. 2019Graph Transformer for Graph-to-Sequence Learningcited by 82
  17. 2018Graph-to-Sequence Learning using Gated Graph Neural Networkscited by 61
  18. 2020Common Sense or World Knowledge? Investigating Adapter-Based Knowledge Injection into Pretrained Transformerscited by 57
  19. 2017Neural AMR: Sequence-to-Sequence Models for Parsing and Generationcited by 56
  20. 2018A Graph-to-Sequence Model for AMR-to-Text Generationcited by 56
  21. 2019Neural data-to-text generation: A comparison between pipeline and end-to-end architecturescited by 51
  22. 2020GPT-too: A language-model-first approach for AMR-to-text generationcited by 38
  23. 2019Semantic Neural Machine Translation using AMRcited by 37
  24. 2018Abstract Meaning Representation for Multi-Document Summarizationcited by 35
  25. 2020Have Your Text and Use It Too! End-to-End Neural Data-to-Text Generation with Semantic Fidelitycited by 35
  26. 2020Modeling Global and Local Node Contexts for Text Generation from Knowledge Graphscited by 30
  27. 2019Densely Connected Graph Convolutional Networks for Graph-to-Sequence Learningcited by 29
  28. 2020Towards a Decomposable Metric for Explainable Evaluation of Text Generation from AMRcited by 25
  29. 2019Enhancing AMR-to-Text Generation with Dual Graph Representationscited by 24
  30. 2020AMR Similarity Metrics from Principlescited by 21
  31. 2019Structural Neural Encoders for AMR-to-text Generationcited by 19
  32. 2019Side-Tuning: A Baseline for Network Adaptation via Additive Side Networkscited by 16
  33. 2020Promoting Graph Awareness in Linearized Graph-to-Text Generationcited by 14
  34. 2017Neural Wikipedian: Generating Textual Summaries from Knowledge Base Triplescited by 13
  35. 2020Online Back-Parsing for AMR-to-Text Generationcited by 9
  36. 2020Lightweight, Dynamic Graph Convolutional Networks for AMR-to-Text Generationcited by 8
  37. 2020Modeling Graph Structure via Relative Position for Text Generation from Knowledge Graphscited by 3

Built on by

Cited by year

Citing papers per year: 2021: 3; 2022: 17; 2023: 12; 2024: 11; 2025: 5.

48 papers.

Sort:

By area:

  1. 2022A Survey of Controllable Text Generation using Transformer-based Pre-trained Language Modelscited by 128
  2. 2022Evaluating and Inducing Personality in Pre-trained Language Modelscited by 122
  3. 2022Pretrained Language Models for Text Generation: A Surveycited by 121
  4. 2021xGQA: Cross-Lingual Visual Question Answeringcited by 72
  5. 2022Graph Pre-training for AMR Parsing and Generationcited by 55
  6. 2024Mitigating Object Hallucination in Large Vision-Language Models via Image-Grounded Guidancecited by 44
  7. 2022FactGraph: Evaluating Factuality in Summarization with Semantic Graph Representationscited by 41
  8. 2024Can Language Beat Numerical Regression? Language-Based Multimodal Trajectory Predictioncited by 31
  9. 2022APPDIA: A Discourse-aware Transformer-based Style Transfer Model for Offensive Social Media Conversationscited by 23
  10. 2023Mixture of Soft Prompts for Controllable Data Generationcited by 23
  11. 2023SMATCH++: Standardized and Extended Evaluation of Semantic Graphscited by 19
  12. 2022GAP: A Graph-aware Language Model Framework for Knowledge Graph-to-Text Generationcited by 16
36 more
  1. 2023Evaluating Generative Models for Graph-to-Text Generationcited by 16
  2. 2024LLM-based multi-agent poetry generation in non-cooperative environmentscited by 15
  3. 2022Paraphrase Identification with Deep Learning: A Review of Datasets and Methodscited by 13
  4. 2024Unifying Structured Data as Graph for Data-to-Text Pre-Trainingcited by 13
  5. 2022Interpretable AMR-Based Question Decomposition for Multi-hop Question Answeringcited by 12
  6. 2021AdapterHub Playground: Simple and Flexible Few-Shot Learning with Adapterscited by 8
  7. 2022Self-supervised Graph Masking Pre-training for Graph-to-Text Generationcited by 8
  8. 2023Incorporating Graph Information in Transformer-based AMR Parsingcited by 7
  9. 2024Evaluating the Smooth Control of Attribute Intensity in Text Generation with LLMscited by 7
  10. 2022UKP-SQUARE: An Online Platform for Question Answering Researchcited by 6
  11. 2022DuNST: Dual Noisy Self Training for Semi-Supervised Controllable Text Generationcited by 5
  12. 2024A Systematic Review of Data-to-Text NLGcited by 4
  13. 2021Smelting Gold and Silver for Improved Multilingual AMR-to-Text Generationcited by 3
  14. 2022Semantic-based Pre-training for Dialogue Understandingcited by 3
  15. 2023CATS: A Pragmatic Chinese Answer-to-Sequence Dataset with Large Scale and High Qualitycited by 3
  16. 2023Text-Only Training for Visual Storytellingcited by 3
  17. 2024Controllable Contextualized Image Captioning: Directing the Visual Narrative through User-Defined Highlightscited by 3
  18. 2024Function-Guided Conditional Generation Using Protein Language Models with Adapterscited by 3
  19. 2025SR-LLM: Rethinking the Structured Representation in Large Language Modelcited by 3
  20. 2022Graph-to-Text Generation with Dynamic Structure Pruningcited by 2
  21. 2022Lexical Complexity Controlled Sentence Generationcited by 2
  22. 2023Investigating the Effect of Relative Positional Embeddings on AMR-to-Text Generation with Structural Adapterscited by 2
  23. 2023Exploiting Abstract Meaning Representation for Open-Domain Question Answeringcited by 2
  24. 2023Few-Shot Data-to-Text Generation via Unified Representation and Multi-Source Learningcited by 2
  25. 2024GraSAME: Injecting Token-Level Structural Information to Pretrained Language Models via Graph-guided Self-Attention Mechanismcited by 2
  26. 2025Refining Interactions: Enhancing Anisotropy in Graph Neural Networks with Language Semanticscited by 2
  27. 2025Survey of Abstract Meaning Representation: Then, Now, Futurecited by 2
  28. 2022Syntax Controlled Knowledge Graph-to-Text Generation with Order and Semantic Consistencycited by 1
  29. 2023XWikiGen: Cross-lingual Summarization for Encyclopedic Text Generation in Low Resource Languagescited by 1
  30. 2023Knowledge Graph for NLG in the context of conversational agentscited by 1
  31. 2023Reranking for Natural Language Generation from Logical Forms: A Study based on Large Language Modelscited by 1
  32. 2024GNNavi: Navigating the Information Flow in Large Language Models by Graph Neural Networkcited by 1
  33. 2022A Survey : Neural Networks for AMR-to-Textcited by 0
  34. 2024CogNarr Ecosystem: Facilitating Group Cognition at Scalecited by 0
  35. 2025SAFT: Structure-Aware Fine-Tuning of LLMs for AMR-to-Text Generationcited by 0
  36. 2025HGAdapter: Hypergraph-based Adapters in Language Models for Code Summarization and Clone Detectioncited by 0