# AMR4NLI: Interpretable and robust NLI measures from semantic graphs

arXiv: https://arxiv.org/abs/2306.00936 · HTML: https://mlsanity.com/paper/2306.00936/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 20 of its 47 references that are in the map.
- First posted on arXiv 1 Jun 2023.

## Builds on (most cited first, 20 of 20)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference](https://mlsanity.com/paper/1704.05426/) (2017)
- [A large annotated corpus for learning natural language inference](https://mlsanity.com/paper/1508.05326/) (2015)
- [BLEURT: Learning Robust Metrics for Text Generation](https://mlsanity.com/paper/2004.04696/) (2020)
- [Annotation Artifacts in Natural Language Inference Data](https://mlsanity.com/paper/1803.02324/) (2018)
- [Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailment](https://mlsanity.com/paper/1907.11932/) (2019)
- [Hypothesis Only Baselines in Natural Language Inference](https://mlsanity.com/paper/1805.01042/) (2018)
- [Probing Neural Network Comprehension of Natural Language Arguments](https://mlsanity.com/paper/1907.07355/) (2019)
- [Learning to Write with Cooperative Discriminators](https://mlsanity.com/paper/1805.06087/) (2018)
- [Look at the First Sentence: Position Bias in Question Answering](https://mlsanity.com/paper/2004.14602/) (2020)
- [FactGraph: Evaluating Factuality in Summarization with Semantic Graph Representations](https://mlsanity.com/paper/2204.06508/) (2022)
- [Examining the Tip of the Iceberg: A Data Set for Idiom Translation](https://mlsanity.com/paper/1802.04681/) (2018)
- [SBERT studies Meaning Representations: Decomposing Sentence Embeddings into Explainable Semantic Features](https://mlsanity.com/paper/2206.07023/) (2022)
- [Evaluating Gender Bias in Natural Language Inference](https://mlsanity.com/paper/2105.05541/) (2021)
- [Towards a Decomposable Metric for Explainable Evaluation of Text Generation from AMR](https://mlsanity.com/paper/2008.08896/) (2020)
- [SMATCH++: Standardized and Extended Evaluation of Semantic Graphs](https://mlsanity.com/paper/2305.06993/) (2023)
- [Weisfeiler-Leman in the BAMBOO: Novel AMR Graph Metrics and a Benchmark for AMR Graph Similarity](https://mlsanity.com/paper/2108.11949/) (2021)
- [Translate, then Parse! A strong baseline for Cross-Lingual AMR Parsing](https://mlsanity.com/paper/2106.04565/) (2021)
- [Better Smatch = Better Parser? AMR evaluation is not so simple anymore](https://mlsanity.com/paper/2210.06461/) (2022)

## Abstract (arXiv)

The task of natural language inference (NLI) asks whether a given premise (expressed in NL) entails a given NL hypothesis. NLI benchmarks contain human ratings of entailment, but the meaning relationships driving these ratings are not formalized. Can the underlying sentence pair relationships be made more explicit in an interpretable yet robust fashion? We compare semantic structures to represent premise and hypothesis, including sets of contextualized embeddings and semantic graphs (Abstract Meaning Representations), and measure whether the hypothesis is a semantic substructure of the premise, utilizing interpretable metrics. Our evaluation on three English benchmarks finds value in both contextualized embeddings and semantic graphs; moreover, they provide complementary signals, and can be leveraged together in a hybrid model.