# Predicting emergent linguistic compositions through time: Syntactic frame extension via multimodal chaining

arXiv: https://arxiv.org/abs/2109.04652 · HTML: https://mlsanity.com/paper/2109.04652/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 49 references that are in the map.
- First posted on arXiv 10 Sep 2021.

## Builds on (most cited first, 13 of 13)

- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [Matching Networks for One Shot Learning](https://mlsanity.com/paper/1606.04080/) (2016)
- [ConceptNet 5.5: An Open Multilingual Graph of General Knowledge](https://mlsanity.com/paper/1612.03975/) (2016)
- [Diachronic Word Embeddings Reveal Statistical Laws of Semantic Change](https://mlsanity.com/paper/1605.09096/) (2016)
- [Grounded Language Learning in a Simulated 3D World](https://mlsanity.com/paper/1706.06551/) (2017)
- [Analysing Lexical Semantic Change with Contextualised Word Representations](https://mlsanity.com/paper/2004.14118/) (2020)
- [Linguistic generalization and compositionality in modern artificial neural networks](https://mlsanity.com/paper/1904.00157/) (2019)
- [Multimodal Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/2011.08899/) (2020)
- [Combining Language and Vision with a Multimodal Skip-gram Model](https://mlsanity.com/paper/1501.02598/) (2015)
- [Learning to Learn to Disambiguate: Meta-Learning for Few-Shot Word Sense Disambiguation](https://mlsanity.com/paper/2004.14355/) (2020)
- [Unsupervised Visual Sense Disambiguation for Verbs using Multimodal Embeddings](https://mlsanity.com/paper/1603.09188/) (2016)
- [End-to-end Deep Prototype and Exemplar Models for Predicting Human Behavior](https://mlsanity.com/paper/2007.08723/) (2020)

## Built on by (most cited first, 3 of 3)

- [Tracing Semantic Variation in Slang](https://mlsanity.com/paper/2210.08635/) (2022)
- [Word sense extension](https://mlsanity.com/paper/2306.05609/) (2023)
- [Systematic word meta-sense extension](https://mlsanity.com/paper/2311.13029/) (2023)

## Abstract (arXiv)

Natural language relies on a finite lexicon to express an unbounded set of emerging ideas. One result of this tension is the formation of new compositions, such that existing linguistic units can be combined with emerging items into novel expressions. We develop a framework that exploits the cognitive mechanisms of chaining and multimodal knowledge to predict emergent compositional expressions through time. We present the syntactic frame extension model (SFEM) that draws on the theory of chaining and knowledge from "percept", "concept", and "language" to infer how verbs extend their frames to form new compositions with existing and novel nouns. We evaluate SFEM rigorously on the 1) modalities of knowledge and 2) categorization models of chaining, in a syntactically parsed English corpus over the past 150 years. We show that multimodal SFEM predicts newly emerged verb syntax and arguments substantially better than competing models using purely linguistic or unimodal knowledge. We find support for an exemplar view of chaining as opposed to a prototype view and reveal how the joint approach of multimodal chaining may be fundamental to the creation of literal and figurative language uses including metaphor and metonymy.