# Maximum Bayes Smatch Ensemble Distillation for AMR Parsing

arXiv: https://arxiv.org/abs/2112.07790 · HTML: https://mlsanity.com/paper/2112.07790/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Builds on 40 of its 62 references that are in the map.
- First posted on arXiv 14 Dec 2021.

## Builds on (most cited first, 30 of 40)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [fairseq: A Fast, Extensible Toolkit for Sequence Modeling](https://mlsanity.com/paper/1904.01038/) (2019)
- [Multilingual Denoising Pre-training for Neural Machine Translation](https://mlsanity.com/paper/2001.08210/) (2020)
- [Sequence-Level Knowledge Distillation](https://mlsanity.com/paper/1606.07947/) (2016)
- [Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidates](https://mlsanity.com/paper/1804.10959/) (2018)
- [Bayesian Deep Learning and a Probabilistic Perspective of Generalization](https://mlsanity.com/paper/2002.08791/) (2020)
- [Multilingual Translation with Extensible Multilingual Pretraining and Finetuning](https://mlsanity.com/paper/2008.00401/) (2020)
- [Investigating Pretrained Language Models for Graph-to-Text Generation](https://mlsanity.com/paper/2007.08426/) (2020)
- [Grammar as a Foreign Language](https://mlsanity.com/paper/1412.7449/) (2014)
- [Neural AMR: Sequence-to-Sequence Models for Parsing and Generation](https://mlsanity.com/paper/1704.08381/) (2017)
- [Leveraging Abstract Meaning Representation for Knowledge Base Question Answering](https://mlsanity.com/paper/2012.01707/) (2020)
- [AMR Parsing as Sequence-to-Graph Transduction](https://mlsanity.com/paper/1905.08704/) (2019)
- [AMR Parsing via Graph-Sequence Iterative Inference](https://mlsanity.com/paper/2004.05572/) (2020)
- [Model Compression with Two-stage Multi-teacher Knowledge Distillation for Web Question Answering System](https://mlsanity.com/paper/1910.08381/) (2019)
- [Selective Knowledge Distillation for Neural Machine Translation](https://mlsanity.com/paper/2105.12967/) (2021)
- [Ensemble Distillation for Neural Machine Translation](https://mlsanity.com/paper/1702.01802/) (2017)
- [GPT-too: A language-model-first approach for AMR-to-text generation](https://mlsanity.com/paper/2005.09123/) (2020)
- [Transition-based Parsing with Stack-Transformers](https://mlsanity.com/paper/2010.10669/) (2020)
- [Broad-Coverage Semantic Parsing as Transduction](https://mlsanity.com/paper/1909.02607/) (2019)
- [Knowledge Distillation for Improved Accuracy in Spoken Question Answering](https://mlsanity.com/paper/2010.11067/) (2020)
- [Improving AMR Parsing with Sequence-to-Sequence Pre-training](https://mlsanity.com/paper/2010.01771/) (2020)
- [Rewarding Smatch: Transition-Based AMR Parsing with Reinforcement Learning](https://mlsanity.com/paper/1905.13370/) (2019)
- [AMR Parsing with Action-Pointer Transformer](https://mlsanity.com/paper/2104.14674/) (2021)
- [Structure-aware Fine-tuning of Sequence-to-sequence Transformers for Transition-based AMR Parsing](https://mlsanity.com/paper/2110.15534/) (2021)
- [Unsupervised Domain Adaptation of a Pretrained Cross-Lingual Language Model](https://mlsanity.com/paper/2011.11499/) (2020)
- [Cross-lingual Abstract Meaning Representation Parsing](https://mlsanity.com/paper/1704.04539/) (2017)

## Built on by (most cited first, 29 of 29)

- [It's MBR All the Way Down: Modern Generation Techniques Through the Lens of Minimum Bayes Risk](https://mlsanity.com/paper/2310.01387/) (2023)
- ["You Are An Expert Linguistic Annotator": Limits of LLMs as Analyzers of Abstract Meaning Representation](https://mlsanity.com/paper/2310.17793/) (2023)
- [Inducing and Using Alignments for Transition-based AMR Parsing](https://mlsanity.com/paper/2205.01464/) (2022)
- [Incorporating Graph Information in Transformer-based AMR Parsing](https://mlsanity.com/paper/2306.13467/) (2023)
- [Ensemble-Instruct: Generating Instruction-Tuning Data with a Heterogeneous Mixture of LMs](https://mlsanity.com/paper/2310.13961/) (2023)
- [Neural Semantic Parsing with Extremely Rich Symbolic Meaning Representations](https://mlsanity.com/paper/2404.12698/) (2024)
- [Neuro-symbolic Commonsense Social Reasoning](https://mlsanity.com/paper/2303.08264/) (2023)
- [On the Role of Summary Content Units in Text Summarization Evaluation](https://mlsanity.com/paper/2404.01701/) (2024)
- [Learning to Transpile AMR into SPARQL](https://mlsanity.com/paper/2112.07877/) (2021)
- [Learning Symbolic Rules over Abstract Meaning Representations for Textual Reinforcement Learning](https://mlsanity.com/paper/2307.02689/) (2023)
- [MASSIVE Multilingual Abstract Meaning Representation: A Dataset and Baselines for Hallucination Detection](https://mlsanity.com/paper/2405.19285/) (2024)
- [A Comparative Study of Neurosymbolic AI Approaches to Interpretable Logical Reasoning](https://mlsanity.com/paper/2508.03366/) (2025)
- [AMR Parsing with Instruction Fine-tuned Pre-trained Language Models](https://mlsanity.com/paper/2304.12272/) (2023)
- [AMRs Assemble! Learning to Ensemble with Autoregressive Models for AMR Parsing](https://mlsanity.com/paper/2306.10786/) (2023)
- [Cross-domain Generalization for AMR Parsing](https://mlsanity.com/paper/2210.12445/) (2022)
- [Identification of Entailment and Contradiction Relations for Natural Language Argumentation: A Neurosymbolic Approach](https://mlsanity.com/paper/2405.01259/) (2024)
- [Making Implicit Premises Explicit in Logical Understanding of Enthymemes](https://mlsanity.com/paper/2603.06114/) (2026)
- [Rematch: Robust and Efficient Matching of Local Knowledge Graphs to Improve Structural and Semantic Similarity](https://mlsanity.com/paper/2404.02126/) (2024)
- [Generating Text from Uniform Meaning Representation](https://mlsanity.com/paper/2502.11973/) (2025)
- [Pairwise Logical Selection of Enthymeme Completions under Semantic-Link Uncertainty](https://mlsanity.com/paper/2608.18820/) (2026)
- [Scalable Learning of Latent Language Structure With Logical Offline Cycle Consistency](https://mlsanity.com/paper/2305.20018/) (2023)
- [FLAG: Financial Long Document Classification via AMR-based GNN](https://mlsanity.com/paper/2410.02024/) (2024)
- [Extracting narrative signals from public discourse: a network-based approach](https://mlsanity.com/paper/2411.00702/) (2024)
- [Evaluation of Finetuned LLMs in AMR Parsing](https://mlsanity.com/paper/2508.05028/) (2025)
- [SETUP: Sentence-level English-To-Uniform Meaning Representation Parser](https://mlsanity.com/paper/2512.07068/) (2025)
- [Object Aligner: A Configurable JSON Schema Similarity Score for Graphs, Applied to LLM Prompt Optimization](https://mlsanity.com/paper/2607.01972/) (2026)
- [Identifying Implicit Premises for Logical Reconstruction of Argument Graphs](https://mlsanity.com/paper/2608.18821/) (2026)
- [How Reliable are AMR Parser Outputs? Uncertainty Estimation for AMR Parsing](https://mlsanity.com/paper/2610.13236/) (2026)
- [Learning Proposition-Level Links from Sentence-Level Supervision for Neuro-Symbolic NLI](https://mlsanity.com/paper/2610.15158/) (2026)

## Abstract (arXiv)

AMR parsing has experienced an unprecendented increase in performance in the last three years, due to a mixture of effects including architecture improvements and transfer learning. Self-learning techniques have also played a role in pushing performance forward. However, for most recent high performant parsers, the effect of self-learning and silver data augmentation seems to be fading. In this paper we propose to overcome this diminishing returns of silver data by combining Smatch-based ensembling techniques with ensemble distillation. In an extensive experimental setup, we push single model English parser performance to a new state-of-the-art, 85.9 (AMR2.0) and 84.3 (AMR3.0), and return to substantial gains from silver data augmentation. We also attain a new state-of-the-art for cross-lingual AMR parsing for Chinese, German, Italian and Spanish. Finally we explore the impact of the proposed technique on domain adaptation, and show that it can produce gains rivaling those of human annotated data for QALD-9 and achieve a new state-of-the-art for BioAMR.