# What company do words keep? Revisiting the distributional semantics of J.R. Firth & Zellig Harris

arXiv: https://arxiv.org/abs/2205.07750 · HTML: https://mlsanity.com/paper/2205.07750/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 151 references that are in the map.
- First posted on arXiv 16 May 2022.

## Builds on (most cited first, 29 of 29)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [Deep contextualized word representations](https://mlsanity.com/paper/1802.05365/) (2018)
- [Enriching Word Vectors with Subword Information](https://mlsanity.com/paper/1607.04606/) (2016)
- [Language (Technology) is Power: A Critical Survey of "Bias" in NLP](https://mlsanity.com/paper/2005.14050/) (2020)
- [Distributed Representations of Sentences and Documents](https://mlsanity.com/paper/1405.4053/) (2014)
- [Speech Recognition with Deep Recurrent Neural Networks](https://mlsanity.com/paper/1303.5778/) (2013)
- [What do you learn from context? Probing for sentence structure in contextualized word representations](https://mlsanity.com/paper/1905.06316/) (2019)
- [Social Bias Frames: Reasoning about Social and Power Implications of Language](https://mlsanity.com/paper/1911.03891/) (2019)
- [Word Embeddings Quantify 100 Years of Gender and Ethnic Stereotypes](https://mlsanity.com/paper/1711.08412/) (2017)
- [Experience Grounds Language](https://mlsanity.com/paper/2004.10151/) (2020)
- [A Persona-Based Neural Conversation Model](https://mlsanity.com/paper/1603.06155/) (2016)
- [Diachronic Word Embeddings Reveal Statistical Laws of Semantic Change](https://mlsanity.com/paper/1605.09096/) (2016)
- [Gender Bias in Neural Natural Language Processing](https://mlsanity.com/paper/1807.11714/) (2018)
- [A Survey of the State of Explainable AI for Natural Language Processing](https://mlsanity.com/paper/2010.00711/) (2020)
- [Diachronic word embeddings and semantic shifts: a survey](https://mlsanity.com/paper/1806.03537/) (2018)
- [The Geometry of Culture: Analyzing Meaning through Word Embeddings](https://mlsanity.com/paper/1803.09288/) (2018)
- [Leveraging Contextual Embeddings for Detecting Diachronic Semantic Shift](https://mlsanity.com/paper/1912.01072/) (2019)
- [A Framework for the Computational Linguistic Analysis of Dehumanization](https://mlsanity.com/paper/2003.03014/) (2020)
- [Characterizing English Variation across Social Media Communities with BERT](https://mlsanity.com/paper/2102.06820/) (2021)
- [Language (Re)modelling: Towards Embodied Language Understanding](https://mlsanity.com/paper/2005.00311/) (2020)
- [(Re)construing Meaning in NLP](https://mlsanity.com/paper/2005.09099/) (2020)
- [The Unstoppable Rise of Computational Linguistics in Deep Learning](https://mlsanity.com/paper/2005.06420/) (2020)
- [Identification of Biased Terms in News Articles by Comparison of Outlet-specific Word Embeddings](https://mlsanity.com/paper/2112.07384/) (2021)
- [Enriching Word Embeddings with Temporal and Spatial Information](https://mlsanity.com/paper/2010.00761/) (2020)

## Built on by (most cited first, 8 of 8)

- [Reducing the Footprint of Multi-Vector Retrieval with Minimal Performance Impact via Token Pooling](https://mlsanity.com/paper/2409.14683/) (2024)
- [Norm of Word Embedding Encodes Information Gain](https://mlsanity.com/paper/2212.09663/) (2022)
- [Can You Learn Semantics Through Next-Word Prediction? The Case of Entailment](https://mlsanity.com/paper/2402.13956/) (2024)
- [From Frege to chatGPT: Compositionality in language, cognition, and deep neural networks](https://mlsanity.com/paper/2405.15164/) (2024)
- [Computational modeling of semantic change](https://mlsanity.com/paper/2304.06337/) (2023)
- [Toward a Critical Toponymy Framework for Named Entity Recognition: A Case Study of Airbnb in New York City](https://mlsanity.com/paper/2310.15302/) (2023)
- [A visual observation on the geometry of UMAP projections of the difference vectors of antonym and synonym word pair embeddings](https://mlsanity.com/paper/2603.24150/) (2026)
- [Gyan: An Explainable Neuro-Symbolic Language Model](https://mlsanity.com/paper/2605.04759/) (2026)

## Abstract (arXiv)

The power of word embeddings is attributed to the linguistic theory that similar words will appear in similar contexts. This idea is specifically invoked by noting that "you shall know a word by the company it keeps," a quote from British linguist J.R. Firth who, along with his American colleague Zellig Harris, is often credited with the invention of "distributional semantics." While both Firth and Harris are cited in all major NLP textbooks and many foundational papers, the content and differences between their theories is seldom discussed. Engaging in a close reading of their work, we discover two distinct and in many ways divergent theories of meaning. One focuses exclusively on the internal workings of linguistic forms, while the other invites us to consider words in new company - not just with other linguistic elements, but also in a broader cultural and situational context. Contrasting these theories from the perspective of current debates in NLP, we discover in Firth a figure who could guide the field towards a more culturally grounded notion of semantics. We consider how an expanded notion of "context" might be modeled in practice through two different strategies: comparative stratification and syntagmatic extension