# Transition-based Abstract Meaning Representation Parsing with Contextual Embeddings

arXiv: https://arxiv.org/abs/2206.06229 · HTML: https://mlsanity.com/paper/2206.06229/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 32 references that are in the map.
- First posted on arXiv 13 Jun 2022.

## Builds on (most cited first, 16 of 16)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation](https://mlsanity.com/paper/1609.08144/) (2016)
- [BERT Rediscovers the Classical NLP Pipeline](https://mlsanity.com/paper/1905.05950/) (2019)
- [What do you learn from context? Probing for sentence structure in contextualized word representations](https://mlsanity.com/paper/1905.06316/) (2019)
- [Linguistic Knowledge and Transferability of Contextual Representations](https://mlsanity.com/paper/1903.08855/) (2019)
- [Simple BERT Models for Relation Extraction and Semantic Role Labeling](https://mlsanity.com/paper/1904.05255/) (2019)
- [Neural AMR: Sequence-to-Sequence Models for Parsing and Generation](https://mlsanity.com/paper/1704.08381/) (2017)
- [Toward Abstractive Summarization Using Semantic Representations](https://mlsanity.com/paper/1805.10399/) (2018)
- [Semantic Neural Machine Translation using AMR](https://mlsanity.com/paper/1902.07282/) (2019)
- [AMR Parsing as Graph Prediction with Latent Alignment](https://mlsanity.com/paper/1805.05286/) (2018)
- [An Incremental Parser for Abstract Meaning Representation](https://mlsanity.com/paper/1608.06111/) (2016)
- [Rewarding Smatch: Transition-Based AMR Parsing with Reinforcement Learning](https://mlsanity.com/paper/1905.13370/) (2019)
- [Text Summarization using Abstract Meaning Representation](https://mlsanity.com/paper/1706.01678/) (2017)

## Abstract (arXiv)

The ability to understand and generate languages sets human cognition apart from other known life forms'. We study a way of combing two of the most successful routes to meaning of language--statistical language models and symbolic semantics formalisms--in the task of semantic parsing. Building on a transition-based, Abstract Meaning Representation (AMR) parser, AmrEager, we explore the utility of incorporating pretrained context-aware word embeddings--such as BERT and RoBERTa--in the problem of AMR parsing, contributing a new parser we dub as AmrBerger. Experiments find these rich lexical features alone are not particularly helpful in improving the parser's overall performance as measured by the SMATCH score when compared to the non-contextual counterpart, while additional concept information empowers the system to outperform the baselines. Through lesion study, we found the use of contextual embeddings helps to make the system more robust against the removal of explicit syntactical features. These findings expose the strength and weakness of the contextual embeddings and the language models in the current form, and motivate deeper understanding thereof.