# A Pattern Language for Machine Learning Tasks

arXiv: https://arxiv.org/abs/2407.02424 · HTML: https://mlsanity.com/paper/2407.02424/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 79 references that are in the map.
- First posted on arXiv 2 Jul 2024.

## Builds on (most cited first, 30 of 34)

- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Prompt-to-Prompt Image Editing with Cross Attention Control](https://mlsanity.com/paper/2208.01626/) (2022)
- [Imagic: Text-Based Real Image Editing with Diffusion Models](https://mlsanity.com/paper/2210.09276/) (2022)
- [Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges](https://mlsanity.com/paper/2104.13478/) (2021)
- [Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks](https://mlsanity.com/paper/1506.03099/) (2015)
- [Generating Sentences from a Continuous Space](https://mlsanity.com/paper/1511.06349/) (2015)
- [Towards a Definition of Disentangled Representations](https://mlsanity.com/paper/1812.02230/) (2018)
- [Neurosymbolic AI: The 3rd Wave](https://mlsanity.com/paper/2012.05876/) (2020)
- [Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Models](https://mlsanity.com/paper/2103.04922/) (2021)
- [Stochastic Variational Video Prediction](https://mlsanity.com/paper/1710.11252/) (2017)
- [TrivialAugment: Tuning-free Yet State-of-the-Art Data Augmentation](https://mlsanity.com/paper/2103.10158/) (2021)
- [A study of the effect of JPG compression on adversarial images](https://mlsanity.com/paper/1608.00853/) (2016)
- [Delete, Retrieve, Generate: A Simple Approach to Sentiment and Style Transfer](https://mlsanity.com/paper/1804.06437/) (2018)
- [Geometry-Consistent Generative Adversarial Networks for One-Sided Unsupervised Domain Mapping](https://mlsanity.com/paper/1809.05852/) (2018)
- [Transforming Delete, Retrieve, Generate Approach for Controlled Text Style Transfer](https://mlsanity.com/paper/1908.09368/) (2019)
- [CycleGAN, a Master of Steganography](https://mlsanity.com/paper/1712.02950/) (2017)
- [A synthetic approach to Markov kernels, conditional independence and theorems on sufficient statistics](https://mlsanity.com/paper/1908.07021/) (2019)
- [Categorical Foundations of Gradient-Based Learning](https://mlsanity.com/paper/2103.01931/) (2021)
- [Loss Functions and Metrics in Deep Learning](https://mlsanity.com/paper/2307.02694/) (2023)
- [Disintegration and Bayesian Inversion via String Diagrams](https://mlsanity.com/paper/1709.00322/) (2017)
- [Neural Probabilistic Logic Programming in Discrete-Continuous Domains](https://mlsanity.com/paper/2303.04660/) (2023)
- [Causal Inference by String Diagram Surgery](https://mlsanity.com/paper/1811.08338/) (2018)
- [StegoGAN: Leveraging Steganography for Non-Bijective Image-to-Image Translation](https://mlsanity.com/paper/2403.20142/) (2024)
- [A survey and taxonomy of loss functions in machine learning](https://mlsanity.com/paper/2301.05579/) (2023)
- [Causal models in string diagrams](https://mlsanity.com/paper/2304.07638/) (2023)
- [Distilling Text into Circuits](https://mlsanity.com/paper/2301.10595/) (2023)
- [Loss as the Inconsistency of a Probabilistic Dependency Graph: Choose Your Model, Not Your Loss Function](https://mlsanity.com/paper/2202.11862/) (2022)
- [Interacting Hopf Algebras](https://mlsanity.com/paper/1403.7048/) (2014)
- [De Finetti's Theorem in Categorical Probability](https://mlsanity.com/paper/2105.02639/) (2021)

## Built on by (most cited first, 4 of 4)

- [Group Diffusion Transformers are Unsupervised Multitask Learners](https://mlsanity.com/paper/2410.15027/) (2024)
- [Logic Explanation of AI Classifiers by Categorical Explaining Functors](https://mlsanity.com/paper/2503.16203/) (2025)
- [From Mechanistic to Compositional Interpretability](https://mlsanity.com/paper/2605.08934/) (2026)
- [A Methodology for Investigating AI Patterns Prevalence in Software Repositories](https://mlsanity.com/paper/2607.00558/) (2026)

## Abstract (arXiv)

We formalise the essential data of objective functions as equality constraints on composites of learners. We call these constraints "tasks", and we investigate the idealised view that such tasks determine model behaviours. We develop a flowchart-like graphical mathematics for tasks that allows us to; (1) offer a unified perspective of approaches in machine learning across domains; (2) design and optimise desired behaviours model-agnostically; and (3) import insights from theoretical computer science into practical machine learning. As a proof-of-concept of the potential practical impact of our theoretical framework, we exhibit and implement a novel "manipulator" task that minimally edits input data to have a desired attribute. Our model-agnostic approach achieves this end-to-end, and without the need for custom architectures, adversarial training, random sampling, or interventions on the data, hence enabling capable, small-scale, and training-stable models.