# Algebraic Machine Learning for Small-to-Medium Datasets Is Competitive against Strong Standard Baselines

arXiv: https://arxiv.org/abs/2605.22155 · HTML: https://mlsanity.com/paper/2605.22155/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 13 of its 65 references that are in the map.
- First posted on arXiv 21 May 2026.

## Builds on (most cited first, 13 of 13)

- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Group Equivariant Convolutional Networks](https://mlsanity.com/paper/1602.07576/) (2016)
- [MedMNIST v2 -- A large-scale lightweight benchmark for 2D and 3D biomedical image classification](https://mlsanity.com/paper/2110.14795/) (2021)
- [Skin Lesion Analysis Toward Melanoma Detection 2018: A Challenge Hosted by the International Skin Imaging Collaboration (ISIC)](https://mlsanity.com/paper/1902.03368/) (2019)
- [Deep Learning for Classical Japanese Literature](https://mlsanity.com/paper/1812.01718/) (2018)
- [Logic Tensor Networks](https://mlsanity.com/paper/2012.13635/) (2020)
- [Logical Neural Networks](https://mlsanity.com/paper/2006.13155/) (2020)
- [Should we really use post-hoc tests based on mean-ranks?](https://mlsanity.com/paper/1505.02288/) (2015)
- [Semantic Embeddings in Semilattices](https://mlsanity.com/paper/2205.12618/) (2022)
- [Algebraic Machine Learning: Learning as computing an algebraic decomposition of a task](https://mlsanity.com/paper/2502.19944/) (2025)

## Abstract (arXiv)

Symbolic methods are generally not considered competitive with strong modern learners on realistic supervised tasks. We evaluate Algebraic Machine Learning (AML), a framework that learns through subdirect decomposition of algebraic structure rather than numerical optimization, against standard baselines on image and tabular classification across varying training-set sizes. We find that AML trained only on training data without using validation or cross-validation outperforms a family of cross-validated baseline methods including CNNs on small to medium image datasets (50--2000 training examples). On tabular datasets in the same size range, XGBoost is overall the best performing method, but AML is nonetheless comparable to methods incorporating task-specific biases such as LightGBM and random forests. AML achieves this competitive performance across two very different types of datasets using a generic algebraic inductive bias, rather than the modality-specific biases built into standard baselines like CNNs for images or XGBoost for tabular data, and requires no cross validation because it has no task-dependent hyperparameters to tune.