# Analogs of Linguistic Structure in Deep Representations

arXiv: https://arxiv.org/abs/1707.08139 · HTML: https://mlsanity.com/paper/1707.08139/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Jul 2017.

## Built on by (most cited first, 7 of 7)

- [Natural Language Descriptions of Deep Visual Features](https://mlsanity.com/paper/2201.11114/) (2022)
- [Emergent Communication of Generalizations](https://mlsanity.com/paper/2106.02668/) (2021)
- [Approximating How Single Head Attention Learns](https://mlsanity.com/paper/2103.07601/) (2021)
- [Linking Emergent and Natural Languages via Corpus Transfer](https://mlsanity.com/paper/2203.13344/) (2022)
- [Semantic HELM: A Human-Readable Memory for Reinforcement Learning](https://mlsanity.com/paper/2306.09312/) (2023)
- [Relations, Negations, and Numbers: Looking for Logic in Generative Text-to-Image Models](https://mlsanity.com/paper/2411.17066/) (2024)
- [PatchGame: Learning to Signal Mid-level Patches in Referential Games](https://mlsanity.com/paper/2111.01785/) (2021)

## Abstract (arXiv)

We investigate the compositional structure of message vectors computed by a deep network trained on a communication game. By comparing truth-conditional representations of encoder-produced message vectors to human-produced referring expressions, we are able to identify aligned (vector, utterance) pairs with the same meaning. We then search for structured relationships among these aligned pairs to discover simple vector space transformations corresponding to negation, conjunction, and disjunction. Our results suggest that neural representations are capable of spontaneously developing a "syntax" with functional analogues to qualitative properties of natural language.