# Improving AMR Parsing with Sequence-to-Sequence Pre-training

arXiv: https://arxiv.org/abs/2010.01771 · HTML: https://mlsanity.com/paper/2010.01771/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Oct 2020.

## Built on by (most cited first, 30 of 30)

- [Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Survey](https://mlsanity.com/paper/2111.01243/) (2021)
- [Graph Pre-training for AMR Parsing and Generation](https://mlsanity.com/paper/2203.07836/) (2022)
- [CLEVE: Contrastive Pre-training for Event Extraction](https://mlsanity.com/paper/2105.14485/) (2021)
- [Maximum Bayes Smatch Ensemble Distillation for AMR Parsing](https://mlsanity.com/paper/2112.07790/) (2021)
- [AMR Parsing with Action-Pointer Transformer](https://mlsanity.com/paper/2104.14674/) (2021)
- [Structure-aware Fine-tuning of Sequence-to-sequence Transformers for Transition-based AMR Parsing](https://mlsanity.com/paper/2110.15534/) (2021)
- [DEAM: Dialogue Coherence Evaluation using AMR-based Semantic Manipulations](https://mlsanity.com/paper/2203.09711/) (2022)
- [SMATCH++: Standardized and Extended Evaluation of Semantic Graphs](https://mlsanity.com/paper/2305.06993/) (2023)
- [Unified Semantic Typing with Meaningful Label Inference](https://mlsanity.com/paper/2205.01826/) (2022)
- [Weisfeiler-Leman in the BAMBOO: Novel AMR Graph Metrics and a Benchmark for AMR Graph Similarity](https://mlsanity.com/paper/2108.11949/) (2021)
- [Translate, then Parse! A strong baseline for Cross-Lingual AMR Parsing](https://mlsanity.com/paper/2106.04565/) (2021)
- [Ensembling Graph Predictions for AMR Parsing](https://mlsanity.com/paper/2110.09131/) (2021)
- [From Paraphrasing to Semantic Parsing: Unsupervised Semantic Parsing via Synchronous Semantic Decoding](https://mlsanity.com/paper/2106.06228/) (2021)
- [Hierarchical Curriculum Learning for AMR Parsing](https://mlsanity.com/paper/2110.07855/) (2021)
- [Better Smatch = Better Parser? AMR evaluation is not so simple anymore](https://mlsanity.com/paper/2210.06461/) (2022)
- [Multilingual AMR Parsing with Noisy Knowledge Distillation](https://mlsanity.com/paper/2109.15196/) (2021)
- [Improved Latent Tree Induction with Distant Supervision via Span Constraints](https://mlsanity.com/paper/2109.05112/) (2021)
- [T-STAR: Truthful Style Transfer using AMR Graph as Intermediate Representation](https://mlsanity.com/paper/2212.01667/) (2022)
- [ATP: AMRize Then Parse! Enhancing AMR Parsing with PseudoAMRs](https://mlsanity.com/paper/2204.08875/) (2022)
- [Strengthening Structural Inductive Biases by Pre-training to Perform Syntactic Transformations](https://mlsanity.com/paper/2407.04543/) (2024)
- [Levi Graph AMR Parser using Heterogeneous Attention](https://mlsanity.com/paper/2107.04152/) (2021)
- [Framing Analysis of Health-Related Narratives: Conspiracy versus Mainstream Media](https://mlsanity.com/paper/2401.10030/) (2024)
- [Making Better Use of Bilingual Information for Cross-Lingual AMR Parsing](https://mlsanity.com/paper/2106.04814/) (2021)
- [AMR Parsing with Causal Hierarchical Attention and Pointers](https://mlsanity.com/paper/2310.11964/) (2023)
- [Adapting Abstract Meaning Representation Parsing to the Clinical Narrative -- the SPRING THYME parser](https://mlsanity.com/paper/2405.09153/) (2024)
- [A Two-Stage Method for Chinese AMR Parsing](https://mlsanity.com/paper/2209.14512/) (2022)
- [Scalable Learning of Latent Language Structure With Logical Offline Cycle Consistency](https://mlsanity.com/paper/2305.20018/) (2023)
- [Widely Interpretable Semantic Representation: Frameless Meaning Representation for Broader Applicability](https://mlsanity.com/paper/2309.06460/) (2023)
- [Guiding AMR Parsing with Reverse Graph Linearization](https://mlsanity.com/paper/2310.08860/) (2023)
- [EventRR: Event Referential Reasoning for Referring Video Object Segmentation](https://mlsanity.com/paper/2508.07171/) (2025)

## Abstract (arXiv)

In the literature, the research on abstract meaning representation (AMR) parsing is much restricted by the size of human-curated dataset which is critical to build an AMR parser with good performance. To alleviate such data size restriction, pre-trained models have been drawing more and more attention in AMR parsing. However, previous pre-trained models, like BERT, are implemented for general purpose which may not work as expected for the specific task of AMR parsing. In this paper, we focus on sequence-to-sequence (seq2seq) AMR parsing and propose a seq2seq pre-training approach to build pre-trained models in both single and joint way on three relevant tasks, i.e., machine translation, syntactic parsing, and AMR parsing itself. Moreover, we extend the vanilla fine-tuning method to a multi-task learning fine-tuning method that optimizes for the performance of AMR parsing while endeavors to preserve the response of pre-trained models. Extensive experimental results on two English benchmark datasets show that both the single and joint pre-trained models significantly improve the performance (e.g., from 71.5 to 80.2 on AMR 2.0), which reaches the state of the art. The result is very encouraging since we achieve this with seq2seq models rather than complex models. We make our code and model available at https://github.com/xdqkid/S2S-AMR-Parser.