mlsanityData to 14 Oct 2026

AMR4NLI: Interpretable and robust NLI measures from semantic graphs

arXiv 2306.00936 ↗First posted 1 Jun 2023cs.CL cs.IRcited by 0

Not on the map. Why?

Abstract (arXiv)

The task of natural language inference (NLI) asks whether a given premise (expressed in NL) entails a given NL hypothesis. NLI benchmarks contain human ratings of entailment, but the meaning relationships driving these ratings are not formalized. Can the underlying sentence pair relationships be made more explicit in an interpretable yet robust fashion? We compare semantic structures to represent premise and hypothesis, including sets of contextualized embeddings and semantic graphs (Abstract Meaning Representations), and measure whether the hypothesis is a semantic substructure of the premise, utilizing interpretable metrics. Our evaluation on three English benchmarks finds value in both contextualized embeddings and semantic graphs; moreover, they provide complementary signals, and can be leveraged together in a hybrid model.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 20 of its 47 references that are in the map. First posted on arXiv 1 Jun 2023.

Builds on

20 of 47 references.

References · 20

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
  3. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,682
  4. 2015A large annotated corpus for learning natural language inferencecited by 1,847
  5. 2020BLEURT: Learning Robust Metrics for Text Generationcited by 1,203
  6. 2018Annotation Artifacts in Natural Language Inference Datacited by 671
  7. 2019Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailmentcited by 671
  8. 2018Hypothesis Only Baselines in Natural Language Inferencecited by 295
  9. 2019Probing Neural Network Comprehension of Natural Language Argumentscited by 214
  10. 2018Learning to Write with Cooperative Discriminatorscited by 119
  11. 2020Look at the First Sentence: Position Bias in Question Answeringcited by 84
  12. 2022FactGraph: Evaluating Factuality in Summarization with Semantic Graph Representationscited by 41
8 more
  1. 2018Examining the Tip of the Iceberg: A Data Set for Idiom Translationcited by 34
  2. 2022SBERT studies Meaning Representations: Decomposing Sentence Embeddings into Explainable Semantic Featurescited by 33
  3. 2021Evaluating Gender Bias in Natural Language Inferencecited by 27
  4. 2020Towards a Decomposable Metric for Explainable Evaluation of Text Generation from AMRcited by 25
  5. 2023SMATCH++: Standardized and Extended Evaluation of Semantic Graphscited by 21
  6. 2021Weisfeiler-Leman in the BAMBOO: Novel AMR Graph Metrics and a Benchmark for AMR Graph Similaritycited by 16
  7. 2021Translate, then Parse! A strong baseline for Cross-Lingual AMR Parsingcited by 15
  8. 2022Better Smatch = Better Parser? AMR evaluation is not so simple anymorecited by 9

Built on by

Not cited yet.