# Multilingual AMR Parsing with Noisy Knowledge Distillation

arXiv: https://arxiv.org/abs/2109.15196 · HTML: https://mlsanity.com/paper/2109.15196/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Placed in Knowledge distillation for LLMs, one of 12 directions in Knowledge distillation.
- Builds on 39 of its 57 references that are in the map.
- First posted on arXiv 30 Sep 2021.

## Placed in

- Knowledge distillation / [Knowledge distillation for LLMs](https://mlsanity.com/distill/knowledge-distillation-for-llms/)

## Builds on (most cited first, 30 of 39)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [FitNets: Hints for Thin Deep Nets](https://mlsanity.com/paper/1412.6550/) (2014)
- [Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfer](https://mlsanity.com/paper/1612.03928/) (2016)
- [Multilingual Denoising Pre-training for Neural Machine Translation](https://mlsanity.com/paper/2001.08210/) (2020)
- [Sequence-Level Knowledge Distillation](https://mlsanity.com/paper/1606.07947/) (2016)
- [On the Variance of the Adaptive Learning Rate and Beyond](https://mlsanity.com/paper/1908.03265/) (2019)
- [XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalization](https://mlsanity.com/paper/2003.11080/) (2020)
- [Do Deep Nets Really Need to be Deep?](https://mlsanity.com/paper/1312.6184/) (2013)
- [Non-Autoregressive Neural Machine Translation](https://mlsanity.com/paper/1711.02281/) (2017)
- [Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyond](https://mlsanity.com/paper/1812.10464/) (2018)
- [Multilingual Translation with Extensible Multilingual Pretraining and Finetuning](https://mlsanity.com/paper/2008.00401/) (2020)
- [XGLUE: A New Benchmark Dataset for Cross-lingual Pre-training, Understanding and Generation](https://mlsanity.com/paper/2004.01401/) (2020)
- [Understanding Knowledge Distillation in Non-autoregressive Machine Translation](https://mlsanity.com/paper/1911.02727/) (2019)
- [Multilingual Neural Machine Translation with Knowledge Distillation](https://mlsanity.com/paper/1902.10461/) (2019)
- [Neural AMR: Sequence-to-Sequence Models for Parsing and Generation](https://mlsanity.com/paper/1704.08381/) (2017)
- [AMR Parsing as Sequence-to-Graph Transduction](https://mlsanity.com/paper/1905.08704/) (2019)
- [AMR Parsing via Graph-Sequence Iterative Inference](https://mlsanity.com/paper/2004.05572/) (2020)
- [Ensemble Distillation for Neural Machine Translation](https://mlsanity.com/paper/1702.01802/) (2017)
- [Semantic Neural Machine Translation using AMR](https://mlsanity.com/paper/1902.07282/) (2019)
- [Abstract Meaning Representation for Multi-Document Summarization](https://mlsanity.com/paper/1806.05655/) (2018)
- [Broad-Coverage Semantic Parsing as Transduction](https://mlsanity.com/paper/1909.02607/) (2019)
- [Noisy Self-Knowledge Distillation for Text Summarization](https://mlsanity.com/paper/2009.07032/) (2020)
- [Improving AMR Parsing with Sequence-to-Sequence Pre-training](https://mlsanity.com/paper/2010.01771/) (2020)
- [Core Semantic First: A Top-down Approach for AMR Parsing](https://mlsanity.com/paper/1909.04303/) (2019)
- [Cross-lingual Abstract Meaning Representation Parsing](https://mlsanity.com/paper/1704.04539/) (2017)

## Built on by (most cited first, 8 of 8)

- [Maximum Bayes Smatch Ensemble Distillation for AMR Parsing](https://mlsanity.com/paper/2112.07790/) (2021)
- [Cultural Adaptation of Recipes](https://mlsanity.com/paper/2310.17353/) (2023)
- [Retrofitting Multilingual Sentence Embeddings with Abstract Meaning Representation](https://mlsanity.com/paper/2210.09773/) (2022)
- [Spanish Abstract Meaning Representation: Annotation of a General Corpus](https://mlsanity.com/paper/2204.07663/) (2022)
- [mPMR: A Multilingual Pre-trained Machine Reader at Scale](https://mlsanity.com/paper/2305.13645/) (2023)
- [Should Cross-Lingual AMR Parsing go Meta? An Empirical Assessment of Meta-Learning and Joint Learning AMR Parsing](https://mlsanity.com/paper/2410.03357/) (2024)
- [Survey of Abstract Meaning Representation: Then, Now, Future](https://mlsanity.com/paper/2505.03229/) (2025)
- [AMR-RE: Abstract Meaning Representations for Retrieval-Based In-Context Learning in Relation Extraction](https://mlsanity.com/paper/2406.10432/) (2024)

## Abstract (arXiv)

We study multilingual AMR parsing from the perspective of knowledge distillation, where the aim is to learn and improve a multilingual AMR parser by using an existing English parser as its teacher. We constrain our exploration in a strict multilingual setting: there is but one model to parse all different languages including English. We identify that noisy input and precise output are the key to successful distillation. Together with extensive pre-training, we obtain an AMR parser whose performances surpass all previously published results on four different foreign languages, including German, Spanish, Italian, and Chinese, by large margins (up to 18.8 textscSmatch points on Chinese and on average 11.3 textscSmatch points). Our parser also achieves comparable performance on English to the latest state-of-the-art English-only parser.