# Learning Functors using Gradient Descent

arXiv: https://arxiv.org/abs/2009.06837 · HTML: https://mlsanity.com/paper/2009.06837/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Sep 2020.

## Built on by (most cited first, 6 of 6)

- [Category Theory in Machine Learning](https://mlsanity.com/paper/2106.07032/) (2021)
- [Reverse Derivative Ascent: A Categorical Approach to Learning Boolean Circuits](https://mlsanity.com/paper/2101.10488/) (2021)
- [Position: LLMs Must Use Functor-Based and RAG-Driven Bias Mitigation for Fairness](https://mlsanity.com/paper/2603.07368/) (2026)
- [Pooling Image Datasets With Multiple Covariate Shift and Imbalance](https://mlsanity.com/paper/2403.02598/) (2024)
- [A Probabilistic Generative Model of Free Categories](https://mlsanity.com/paper/2205.04545/) (2022)
- [Coalgebras for categorical deep learning: Representability and universal approximation](https://mlsanity.com/paper/2603.03227/) (2026)

## Abstract (arXiv)

Neural networks are a general framework for differentiable optimization which includes many other machine learning approaches as special cases. In this paper we build a category-theoretic formalism around a neural network system called CycleGAN. CycleGAN is a general approach to unpaired image-to-image translation that has been getting attention in the recent years. Inspired by categorical database systems, we show that CycleGAN is a "schema", i.e. a specific category presented by generators and relations, whose specific parameter instantiations are just set-valued functors on this schema. We show that enforcing cycle-consistencies amounts to enforcing composition invariants in this category. We generalize the learning procedure to arbitrary such categories and show a special class of functors, rather than functions, can be learned using gradient descent. Using this framework we design a novel neural network system capable of learning to insert and delete objects from images without paired data. We qualitatively evaluate the system on the CelebA dataset and obtain promising results.