# Symbolic, Distributed and Distributional Representations for Natural Language Processing in the Era of Deep Learning: a Survey

arXiv: https://arxiv.org/abs/1702.00764 · HTML: https://mlsanity.com/paper/1702.00764/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Feb 2017.

## Built on by (most cited first, 8 of 8)

- [A Survey of Text Representation Methods and Their Genealogy](https://mlsanity.com/paper/2211.14591/) (2022)
- [Resolving Regular Polysemy in Named Entities](https://mlsanity.com/paper/2401.09758/) (2024)
- [Lacking the embedding of a word? Look it up into a traditional dictionary](https://mlsanity.com/paper/2109.11763/) (2021)
- [Synergizing Machine Learning & Symbolic Methods: A Survey on Hybrid Approaches to Natural Language Processing](https://mlsanity.com/paper/2401.11972/) (2024)
- [Compositional Structures in Neural Embedding and Interaction Decompositions](https://mlsanity.com/paper/2407.08934/) (2024)
- [Quality Assurance of Generative Dialog Models in an Evolving Conversational Agent Used for Swedish Language Practice](https://mlsanity.com/paper/2203.15414/) (2022)
- [Unbiased Weight Maximization](https://mlsanity.com/paper/2307.13270/) (2023)
- [R2-CoD: Understanding Text-Graph Complementarity in Relational Reasoning via Knowledge Co-Distillation](https://mlsanity.com/paper/2508.01475/) (2025)

## Abstract (arXiv)

Natural language is inherently a discrete symbolic representation of human knowledge. Recent advances in machine learning (ML) and in natural language processing (NLP) seem to contradict the above intuition: discrete symbols are fading away, erased by vectors or tensors called distributed and distributional representations. However, there is a strict link between distributed/distributional representations and discrete symbols, being the first an approximation of the second. A clearer understanding of the strict link between distributed/distributional representations and symbols may certainly lead to radically new deep learning networks. In this paper we make a survey that aims to renew the link between symbolic representations and distributed/distributional representations. This is the right time to revitalize the area of interpreting how discrete symbols are represented inside neural networks.