# Learnware: Small Models Do Big

arXiv: https://arxiv.org/abs/2210.03647 · HTML: https://mlsanity.com/paper/2210.03647/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 32 references that are in the map.
- First posted on arXiv 7 Oct 2022.

## Builds on (most cited first, 13 of 13)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [A Survey on Multi-Task Learning](https://mlsanity.com/paper/1707.08114/) (2017)
- [Kernel Mean Embedding of Distributions: A Review and Beyond](https://mlsanity.com/paper/1605.09522/) (2016)
- [Domain Agnostic Learning with Disentangled Representations](https://mlsanity.com/paper/1904.12347/) (2019)
- [Mixture Proportion Estimation via Kernel Embedding of Distributions](https://mlsanity.com/paper/1603.02501/) (2016)
- [Open-environment Machine Learning](https://mlsanity.com/paper/2206.00423/) (2022)
- [Class-prior Estimation for Learning from Positive and Unlabeled Data](https://mlsanity.com/paper/1611.01586/) (2016)
- [Near-Optimal Coresets of Kernel Density Estimates](https://mlsanity.com/paper/1802.01751/) (2018)
- [Discrepancy, Coresets, and Sketches in Machine Learning](https://mlsanity.com/paper/1906.04845/) (2019)
- [Handling Concept Drift via Model Reuse](https://mlsanity.com/paper/1809.02804/) (2018)
- [Model Reuse with Reduced Kernel Mean Embedding Specification](https://mlsanity.com/paper/2001.07135/) (2020)

## Built on by (most cited first, 16 of 16)

- [A Survey on Large Language Models for Mathematical Reasoning](https://mlsanity.com/paper/2506.08446/) (2025)
- [Representation Learning for Tabular Data: A Comprehensive Survey](https://mlsanity.com/paper/2504.16109/) (2025)
- [Model Spider: Learning to Rank Pre-Trained Models Efficiently](https://mlsanity.com/paper/2306.03900/) (2023)
- [Foundation Models Meet Visualizations: Challenges and Opportunities](https://mlsanity.com/paper/2310.05771/) (2023)
- [Wings: Learning Multimodal LLMs without Text-only Forgetting](https://mlsanity.com/paper/2406.03496/) (2024)
- [Beimingwu: A Learnware Dock System](https://mlsanity.com/paper/2401.14427/) (2024)
- [Cooperation of Experts: Fusing Heterogeneous Information with Large Margin](https://mlsanity.com/paper/2505.20853/) (2025)
- [A Lightweight Measure of Classification Difficulty from Application Dataset Characteristics](https://mlsanity.com/paper/2404.05981/) (2024)
- [Vision-Language Model Selection and Reuse for Downstream Adaptation](https://mlsanity.com/paper/2501.18271/) (2025)
- [SeqFusion: Sequential Fusion of Pre-Trained Models for Zero-Shot Time-Series Forecasting](https://mlsanity.com/paper/2503.02836/) (2025)
- [Learnware of Language Models: Specialized Small Language Models Can Do Big](https://mlsanity.com/paper/2505.13425/) (2025)
- [Bridging Today and the Future of Humanity: AI Safety in 2024 and Beyond](https://mlsanity.com/paper/2410.18114/) (2024)
- [CGI: Identifying Conditional Generative Models with Example Images](https://mlsanity.com/paper/2501.13991/) (2025)
- [Efficient Neural Network Model Selection for Few-Class Application Datasets](https://mlsanity.com/paper/2606.19712/) (2026)
- [Learnware for CSI Feedback: Scene-specific Small Models Can Do Big](https://mlsanity.com/paper/2608.17760/) (2026)
- [Learnware and AI Model Management System](https://mlsanity.com/paper/2609.11656/) (2026)

## Abstract (arXiv)

There are complaints about current machine learning techniques such as the requirement of a huge amount of training data and proficient training skills, the difficulty of continual learning, the risk of catastrophic forgetting, the leaking of data privacy/proprietary, etc. Most research efforts have been focusing on one of those concerned issues separately, paying less attention to the fact that most issues are entangled in practice. The prevailing big model paradigm, which has achieved impressive results in natural language processing and computer vision applications, has not yet addressed those issues, whereas becoming a serious source of carbon emissions. This article offers an overview of the learnware paradigm, which attempts to enable users not need to build machine learning models from scratch, with the hope of reusing small models to do things even beyond their original purposes, where the key ingredient is the specification which enables a trained model to be adequately identified to reuse according to the requirement of future users who know nothing about the model in advance.