# Learning Functional Distributional Semantics with Visual Data

arXiv: https://arxiv.org/abs/2204.10624 · HTML: https://mlsanity.com/paper/2204.10624/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 44 references that are in the map.
- First posted on arXiv 22 Apr 2022.

## Builds on (most cited first, 20 of 20)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [VQA: Visual Question Answering](https://mlsanity.com/paper/1505.00468/) (2015)
- [Deep contextualized word representations](https://mlsanity.com/paper/1802.05365/) (2018)
- [Deep Visual-Semantic Alignments for Generating Image Descriptions](https://mlsanity.com/paper/1412.2306/) (2014)
- [The Neuro-Symbolic Concept Learner: Interpreting Scenes, Words, and Sentences From Natural Supervision](https://mlsanity.com/paper/1904.12584/) (2019)
- [Multimodal Compact Bilinear Pooling for Visual Question Answering and Visual Grounding](https://mlsanity.com/paper/1606.01847/) (2016)
- [Unifying Visual-Semantic Embeddings with Multimodal Neural Language Models](https://mlsanity.com/paper/1411.2539/) (2014)
- [SimLex-999: Evaluating Semantic Models with (Genuine) Similarity Estimation](https://mlsanity.com/paper/1408.3456/) (2014)
- [Visual Concept-Metaconcept Learning](https://mlsanity.com/paper/2002.01464/) (2020)
- [Semi-analytical approximations to statistical moments of sigmoid and softmax mappings of normal variables](https://mlsanity.com/paper/1703.00091/) (2017)
- [What are the Goals of Distributional Semantics?](https://mlsanity.com/paper/2005.02982/) (2020)
- [Functional Distributional Semantics](https://mlsanity.com/paper/1606.08003/) (2016)
- [Efficient Multi-Modal Embeddings from Structured Data](https://mlsanity.com/paper/2110.02577/) (2021)
- [Autoencoding Pixies: Amortised Variational Inference with Graph Convolutions for Functional Distributional Semantics](https://mlsanity.com/paper/2005.02991/) (2020)
- [Semantic Composition via Probabilistic Model Theory](https://mlsanity.com/paper/1709.00226/) (2017)
- [Natural Language Semantics With Pictures: Some Language & Vision Datasets and Potential Uses for Computational Semantics](https://mlsanity.com/paper/1904.07318/) (2019)
- [Linguists Who Use Probabilistic Models Love Them: Quantification in Functional Distributional Semantics](https://mlsanity.com/paper/2006.03002/) (2020)

## Built on by (most cited first, 1 of 1)

- [Visual Spatial Reasoning](https://mlsanity.com/paper/2205.00363/) (2022)

## Abstract (arXiv)

Functional Distributional Semantics is a recently proposed framework for learning distributional semantics that provides linguistic interpretability. It models the meaning of a word as a binary classifier rather than a numerical vector. In this work, we propose a method to train a Functional Distributional Semantics model with grounded visual data. We train it on the Visual Genome dataset, which is closer to the kind of data encountered in human language acquisition than a large text corpus. On four external evaluation datasets, our model outperforms previous work on learning semantics from Visual Genome.