# Two Discourse Driven Language Models for Semantics

arXiv: https://arxiv.org/abs/1606.05679 · HTML: https://mlsanity.com/paper/1606.05679/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Jun 2016.

## Built on by (most cited first, 8 of 8)

- [SUGARCREPE++ Dataset: Vision-Language Model Sensitivity to Semantic and Lexical Alterations](https://mlsanity.com/paper/2406.11171/) (2024)
- [The Future is not One-dimensional: Complex Event Schema Induction by Graph Modeling for Event Prediction](https://mlsanity.com/paper/2104.06344/) (2021)
- [GraphPlan: Story Generation by Planning with Event Graph](https://mlsanity.com/paper/2102.02977/) (2021)
- [EtriCA: Event-Triggered Context-Aware Story Generation Augmented by Cross Attention](https://mlsanity.com/paper/2210.12463/) (2022)
- [Human-in-the-Loop Schema Induction](https://mlsanity.com/paper/2302.13048/) (2023)
- [Open-Vocabulary Argument Role Prediction for Event Extraction](https://mlsanity.com/paper/2211.01577/) (2022)
- [Zero-Shot On-the-Fly Event Schema Induction](https://mlsanity.com/paper/2210.06254/) (2022)
- [A Cross-Attention Augmented Model for Event-Triggered Context-Aware Story Generation](https://mlsanity.com/paper/2311.11271/) (2023)

## Abstract (arXiv)

Natural language understanding often requires deep semantic knowledge. Expanding on previous proposals, we suggest that some important aspects of semantic knowledge can be modeled as a language model if done at an appropriate level of abstraction. We develop two distinct models that capture semantic frame chains and discourse information while abstracting over the specific mentions of predicates and entities. For each model, we investigate four implementations: a "standard" N-gram language model and three discriminatively trained "neural" language models that generate embeddings for semantic frames. The quality of the semantic language models (SemLM) is evaluated both intrinsically, using perplexity and a narrative cloze test and extrinsically - we show that our SemLM helps improve performance on semantic natural language processing tasks such as co-reference resolution and discourse parsing.