mlsanityData to 14 Oct 2026

Graph Pre-training for AMR Parsing and Generation

arXiv 2203.07836 ↗First posted 15 Mar 2022cs.CLcited by 55

Not on the map. Why?

Abstract (arXiv)

Abstract meaning representation (AMR) highlights the core semantic information of text in a graph structure. Recently, pre-trained language models (PLMs) have advanced tasks of AMR parsing and AMR-to-text generation, respectively. However, PLMs are typically pre-trained on textual data, thus are sub-optimal for modeling structural knowledge. To this end, we investigate graph self-supervised training to improve the structure awareness of PLMs over AMR graphs. In particular, we introduce two graph auto-encoding strategies for graph-to-graph pre-training and four tasks to integrate text and graph information during pre-training. We further design a unified framework to bridge the gap between pre-training and fine-tuning tasks. Experiments on both AMR parsing and AMR-to-text generation show the superiority of our model. To our knowledge, we are the first to consider pre-training on semantic graphs.

Key facts. Cited by 55 papers in the map (data through 14 Oct 2026). Builds on 36 of its 64 references that are in the map. First posted on arXiv 15 Mar 2022.

Builds on

36 of 64 references.

References · 36

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  4. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
  5. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
  6. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,548
  7. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
  8. 2016Variational Graph Auto-Encoderscited by 1,382
  9. 2019Strategies for Pre-training Graph Neural Networkscited by 971
  10. 2020GPT-GNN: Generative Pre-Training of Graph Neural Networkscited by 303
  11. 2021DialogSum: A Real-Life Scenario Dialogue Summarization Datasetcited by 183
  12. 2020Investigating Pretrained Language Models for Graph-to-Text Generationcited by 117
24 more
  1. 2018Graph-to-Sequence Learning using Gated Graph Neural Networkscited by 61
  2. 2017Neural AMR: Sequence-to-Sequence Models for Parsing and Generationcited by 56
  3. 2018A Graph-to-Sequence Model for AMR-to-Text Generationcited by 56
  4. 2019AMR Parsing as Sequence-to-Graph Transductioncited by 50
  5. 2021Structural Adapters in Pretrained Language Models for AMR-to-text Generationcited by 48
  6. 2020AMR Parsing via Graph-Sequence Iterative Inferencecited by 47
  7. 2018Toward Abstractive Summarization Using Semantic Representationscited by 43
  8. 2020GPT-too: A language-model-first approach for AMR-to-text generationcited by 38
  9. 2020Transition-based Parsing with Stack-Transformerscited by 38
  10. 2019Semantic Neural Machine Translation using AMRcited by 37
  11. 2018AMR Parsing as Graph Prediction with Latent Alignmentcited by 36
  12. 2018Abstract Meaning Representation for Multi-Document Summarizationcited by 35
  13. 2019Modeling Graph Structure in Transformer for Better AMR-to-Text Generationcited by 35
  14. 2019Broad-Coverage Semantic Parsing as Transductioncited by 32
  15. 2020Improving AMR Parsing with Sequence-to-Sequence Pre-trainingcited by 30
  16. 2021AMR Parsing with Action-Pointer Transformercited by 25
  17. 2021Semantic Representation for Dialogue Modelingcited by 24
  18. 2019Structural Neural Encoders for AMR-to-text Generationcited by 19
  19. 2017AMR Parsing using Stack-LSTMscited by 15
  20. 2018An AMR Aligner Tuned by Transition-based Parsercited by 12
  21. 2017Addressing the Data Sparsity Issue in Neural AMR Parsingcited by 11
  22. 2020Online Back-Parsing for AMR-to-Text Generationcited by 9
  23. 2020Lightweight, Dynamic Graph Convolutional Networks for AMR-to-Text Generationcited by 8
  24. 2020A Differentiable Relaxation of Graph Segmentation and Alignment for AMR Parsingcited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 9; 2023: 19; 2024: 15; 2025: 11; 2026: 1.

55 papers.

Sort:

By area:

  1. 2023FACTUAL: A Benchmark for Faithful and Consistent Textual Scene Graph Parsingcited by 50
  2. 2024Don't Forget to Connect! Improving RAG with Graph-based Rerankingcited by 49
  3. 2025FineCIR: Explicit Parsing of Fine-Grained Modification Semantics for Composed Image Retrievalcited by 42
  4. 2023AMPERE: AMR-Aware Prefix for Generation-Based Event Argument Extraction Modelcited by 19
  5. 2024ReLiK: Retrieve and LinK, Fast and Accurate Entity Linking and Relation Extraction on an Academic Budgetcited by 16
  6. 2023"You Are An Expert Linguistic Annotator": Limits of LLMs as Analyzers of Abstract Meaning Representationcited by 15
  7. 2022BenchCLAMP: A Benchmark for Evaluating Language Models on Syntactic and Semantic Parsingcited by 13
  8. 2024Unifying Structured Data as Graph for Data-to-Text Pre-Trainingcited by 13
  9. 2022Inducing and Using Alignments for Transition-based AMR Parsingcited by 12
  10. 2023Pre-Trained Language-Meaning Models for Multilingual Parsing and Generationcited by 10
  11. 2023An AMR-based Link Prediction Approach for Document-level Event Argument Extractioncited by 8
  12. 2024ABEX: Data Augmentation for Low-Resource NLU via Expanding Abstract Descriptionscited by 8
38 more
  1. 2023Incorporating Graph Information in Transformer-based AMR Parsingcited by 7
  2. 2022T-STAR: Truthful Style Transfer using AMR Graph as Intermediate Representationcited by 6
  3. 2023AMR Parsing is Far from Solved: GrAPES, the Granular AMR Parsing Evaluation Suitecited by 6
  4. 2024Neural Semantic Parsing with Extremely Rich Symbolic Meaning Representationscited by 6
  5. 2022Retrofitting Multilingual Sentence Embeddings with Abstract Meaning Representationcited by 5
  6. 2025Entailment-Preserving First-order Logic Representations in Natural Language Entailmentcited by 5
  7. 2023Causal Document-Grounded Dialogue Pre-trainingcited by 4
  8. 2024A Systematic Review of Data-to-Text NLGcited by 4
  9. 2024Semantic Graphs for Syntactic Simplification: A Revisit from the Age of LLMcited by 4
  10. 2022Semantic-based Pre-training for Dialogue Understandingcited by 3
  11. 2023AMR Parsing with Instruction Fine-tuned Pre-trained Language Modelscited by 3
  12. 2023AMRs Assemble! Learning to Ensemble with Autoregressive Models for AMR Parsingcited by 3
  13. 2024Prompting for Numerical Sequences: A Case Study on Market Comment Generationcited by 3
  14. 2024MACT: Model-Agnostic Cross-Lingual Training for Discourse Representation Structure Parsingcited by 3
  15. 2025SR-LLM: Rethinking the Structured Representation in Large Language Modelcited by 3
  16. 2022Cross-domain Generalization for AMR Parsingcited by 2
  17. 2022An Empirical Revisiting of Linguistic Knowledge Fusion in Language Understanding Taskscited by 2
  18. 2023Exploiting Abstract Meaning Representation for Open-Domain Question Answeringcited by 2
  19. 2023Hi-ArG: Exploring the Integration of Hierarchical Argumentation Graphs in Language Pretrainingcited by 2
  20. 2024Retrieval-Augmented Semantic Parsing: Improving Generalization with Lexical Knowledgecited by 2
  21. 2025Survey of Abstract Meaning Representation: Then, Now, Futurecited by 2
  22. 2023Unleashing the True Potential of Sequence-to-Sequence Models for Sequence Tagging and Structure Parsingcited by 1
  23. 2023NLP Workbench: Efficient and Extensible Integration of State-of-the-art Text Mining Toolscited by 1
  24. 2023AMR Parsing with Causal Hierarchical Attention and Pointerscited by 1
  25. 2024Rematch: Robust and Efficient Matching of Local Knowledge Graphs to Improve Structural and Semantic Similaritycited by 1
  26. 2024Adapting Abstract Meaning Representation Parsing to the Clinical Narrative -- the SPRING THYME parsercited by 1
  27. 2024Handling Ontology Gaps in Semantic Parsingcited by 1
  28. 2025Generating Text from Uniform Meaning Representationcited by 1
  29. 2025A Theorem-Proving-Based Evaluation of Neural Semantic Parsingcited by 1
  30. 2022Cross-lingual AMR Aligner: Paying Attention to Cross-Attentioncited by 0
  31. 2022A Two-Stage Method for Chinese AMR Parsingcited by 0
  32. 2023Scalable Learning of Latent Language Structure With Logical Offline Cycle Consistencycited by 0
  33. 2023Leveraging Denoised Abstract Meaning Representation for Grammatical Error Correctioncited by 0
  34. 2023Guiding AMR Parsing with Reverse Graph Linearizationcited by 0
  35. 2023Rethinking Relation Classification with Graph Meaning Representationscited by 0
  36. 2024X-AMR Annotation Toolcited by 0
  37. 2024Modeling Unified Semantic Discourse Structure for High-quality Headline Generationcited by 0
  38. 2025SAFT: Structure-Aware Fine-Tuning of LLMs for AMR-to-Text Generationcited by 0

50 most cited of 55 All 55 (JSON)