# Categorical Representation Learning: Morphism is All You Need

arXiv: https://arxiv.org/abs/2103.14770 · HTML: https://mlsanity.com/paper/2103.14770/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 3 of its 7 references that are in the map.
- First posted on arXiv 26 Mar 2021.

## Builds on (most cited first, 3 of 3)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Graph Neural Networks: A Review of Methods and Applications](https://mlsanity.com/paper/1812.08434/) (2018)

## Built on by (most cited first, 6 of 6)

- [Artificial General Intelligence (AGI)-Native Wireless Systems: A Journey Beyond 6G](https://mlsanity.com/paper/2405.02336/) (2024)
- [Noncommutative Geometry of Computational Models and Uniformization for Framed Quiver Varieties](https://mlsanity.com/paper/2201.05900/) (2022)
- [Categorical Representation Learning and RG flow operators for algorithmic classifiers](https://mlsanity.com/paper/2203.07975/) (2022)
- [DeepProphet2 -- A Deep Learning Gene Recommendation Engine](https://mlsanity.com/paper/2208.01918/) (2022)
- [The Gauss-Markov Adjunction Provides Categorical Semantics of Residuals in Supervised Learning](https://mlsanity.com/paper/2507.02442/) (2025)
- [Disentangling Hyperedges through the Lens of Category Theory](https://mlsanity.com/paper/2510.16289/) (2025)

## Abstract (arXiv)

We provide a construction for categorical representation learning and introduce the foundations of "textitcategorifier". The central theme in representation learning is the idea of everything to vector. Every object in a dataset S can be represented as a vector in ℝn by an textitencoding map E: Obj(S)→ℝn. More importantly, every morphism can be represented as a matrix E: Hom(S)→ℝnn. The encoding map E is generally modeled by a textitdeep neural network. The goal of representation learning is to design appropriate tasks on the dataset to train the encoding map (assuming that an encoding is optimal if it universally optimizes the performance on various tasks). However, the latter is still a textitset-theoretic approach. The goal of the current article is to promote the representation learning to a new level via a textitcategory-theoretic approach. As a proof of concept, we provide an example of a text translator equipped with our technology, showing that our categorical learning model outperforms the current deep learning models by 17 times. The content of the current article is part of the recent US patent proposal (patent application number: 63110906).