# Cross-lingual AMR Aligner: Paying Attention to Cross-Attention

arXiv: https://arxiv.org/abs/2206.07587 · HTML: https://mlsanity.com/paper/2206.07587/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 33 of its 55 references that are in the map.
- First posted on arXiv 15 Jun 2022.

## Builds on (most cited first, 30 of 33)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://mlsanity.com/paper/1312.6034/) (2013)
- [Visualizing and Understanding Convolutional Networks](https://mlsanity.com/paper/1311.2901/) (2013)
- [Deep contextualized word representations](https://mlsanity.com/paper/1802.05365/) (2018)
- [Learning Important Features Through Propagating Activation Differences](https://mlsanity.com/paper/1704.02685/) (2017)
- [BERT Rediscovers the Classical NLP Pipeline](https://mlsanity.com/paper/1905.05950/) (2019)
- [Multilingual Denoising Pre-training for Neural Machine Translation](https://mlsanity.com/paper/2001.08210/) (2020)
- [Striving for Simplicity: The All Convolutional Net](https://mlsanity.com/paper/1412.6806/) (2014)
- [Captum: A unified and generic model interpretability library for PyTorch](https://mlsanity.com/paper/2009.07896/) (2020)
- [From Softmax to Sparsemax: A Sparse Model of Attention and Multi-Label Classification](https://mlsanity.com/paper/1602.02068/) (2016)
- [Word Alignment by Fine-tuning Embeddings on Parallel Corpora](https://mlsanity.com/paper/2101.08231/) (2021)
- [The elephant in the interpretability room: Why use attention as explanation when we have saliency methods?](https://mlsanity.com/paper/2010.05607/) (2020)
- [Attention Interpretability Across NLP Tasks](https://mlsanity.com/paper/1909.11218/) (2019)
- [Graph Pre-training for AMR Parsing and Generation](https://mlsanity.com/paper/2203.07836/) (2022)
- [Leveraging Abstract Meaning Representation for Knowledge Base Question Answering](https://mlsanity.com/paper/2012.01707/) (2020)
- [Improving Natural Language Processing Tasks with Human Gaze-Guided Neural Attention](https://mlsanity.com/paper/2010.07891/) (2020)
- [Semantic Neural Machine Translation using AMR](https://mlsanity.com/paper/1902.07282/) (2019)
- [Abstract Meaning Representation for Multi-Document Summarization](https://mlsanity.com/paper/1806.05655/) (2018)
- [Supervising Model Attention with Human Explanations for Robust Natural Language Inference](https://mlsanity.com/paper/2104.08142/) (2021)
- [Structure-aware Fine-tuning of Sequence-to-sequence Transformers for Transition-based AMR Parsing](https://mlsanity.com/paper/2110.15534/) (2021)
- [Guided Neural Language Generation for Abstractive Summarization using Abstract Meaning Representation](https://mlsanity.com/paper/1808.09160/) (2018)
- [Long-Span Summarization via Local Attention and Content Selection](https://mlsanity.com/paper/2105.03801/) (2021)
- [Do Context-Aware Translation Models Pay the Right Attention?](https://mlsanity.com/paper/2105.06977/) (2021)
- [Mask-Align: Self-Supervised Neural Word Alignment](https://mlsanity.com/paper/2012.07162/) (2020)
- [Cross-lingual Abstract Meaning Representation Parsing](https://mlsanity.com/paper/1704.04539/) (2017)
- [Modeling Concentrated Cross-Attention for Neural Machine Translation with Gaussian Mixture Model](https://mlsanity.com/paper/2109.05244/) (2021)
- [Translate, then Parse! A strong baseline for Cross-Lingual AMR Parsing](https://mlsanity.com/paper/2106.04565/) (2021)
- [Guiding Attention for Self-Supervised Learning with Transformers](https://mlsanity.com/paper/2010.02399/) (2020)
- [An AMR Aligner Tuned by Transition-based Parser](https://mlsanity.com/paper/1810.03541/) (2018)

## Abstract (arXiv)

This paper introduces a novel aligner for Abstract Meaning Representation (AMR) graphs that can scale cross-lingually, and is thus capable of aligning units and spans in sentences of different languages. Our approach leverages modern Transformer-based parsers, which inherently encode alignment information in their cross-attention weights, allowing us to extract this information during parsing. This eliminates the need for English-specific rules or the Expectation Maximization (EM) algorithm that have been used in previous approaches. In addition, we propose a guided supervised method using alignment to further enhance the performance of our aligner. We achieve state-of-the-art results in the benchmarks for AMR alignment and demonstrate our aligner's ability to obtain them across multiple languages. Our code will be available at hrefhttps://www.github.com/Babelscape/AMR-alignmentgithub.com/Babelscape/AMR-alignment.