# Functorial Language Models

arXiv: https://arxiv.org/abs/2103.14411 · HTML: https://mlsanity.com/paper/2103.14411/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 35 references that are in the map.
- First posted on arXiv 26 Mar 2021.

## Builds on (most cited first, 13 of 13)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Foundations for Near-Term Quantum Natural Language Processing](https://mlsanity.com/paper/2012.03755/) (2020)
- [Quantum Natural Language Processing on Near-Term Quantum Computers](https://mlsanity.com/paper/2005.04147/) (2020)
- [Open System Categorical Quantum Semantics in Natural Language Processing](https://mlsanity.com/paper/1502.00831/) (2015)
- [Lambek vs. Lambek: Functorial Vector Space Semantics and String Diagrams for Lambek Calculus](https://mlsanity.com/paper/1302.0393/) (2013)
- [Sentence Entailment in Compositional Distributional Semantics](https://mlsanity.com/paper/1512.04419/) (2015)
- [Reasoning about Meaning in Natural Language with Compact Closed Categories and Frobenius Algebras](https://mlsanity.com/paper/1401.5980/) (2014)
- [Diagrammatic Differentiation for Quantum Machine Learning](https://mlsanity.com/paper/2103.07960/) (2021)
- [Towards Compositional Distributional Discourse Analysis](https://mlsanity.com/paper/1811.03277/) (2018)
- [Functorial Question Answering](https://mlsanity.com/paper/1905.07408/) (2019)
- [Autonomization of Monoidal Categories](https://mlsanity.com/paper/1411.3827/) (2014)
- [Exploring Semantic Incrementality with Dynamic Syntax and Vector Space Semantics](https://mlsanity.com/paper/1811.00614/) (2018)
- [Incremental Monoidal Grammars](https://mlsanity.com/paper/2001.02296/) (2020)

## Built on by (most cited first, 1 of 1)

- [Model Editing as a Robust and Denoised variant of DPO: A Case Study on Toxicity](https://mlsanity.com/paper/2405.13967/) (2024)

## Abstract (arXiv)

We introduce functorial language models: a principled way to compute probability distributions over word sequences given a monoidal functor from grammar to meaning. This yields a method for training categorical compositional distributional (DisCoCat) models on raw text data. We provide a proof-of-concept implementation in DisCoPy, the Python toolbox for monoidal categories.