# Meta-Learning Update Rules for Unsupervised Representation Learning

arXiv: https://arxiv.org/abs/1804.00222 · HTML: https://mlsanity.com/paper/1804.00222/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Mar 2018.

## Built on by (most cited first, 30 of 37)

- [Learning to (Learn at Test Time): RNNs with Expressive Hidden States](https://mlsanity.com/paper/2407.04620/) (2024)
- [Investigating Bi-Level Optimization for Learning and Vision from a Unified Perspective: A Survey and Beyond](https://mlsanity.com/paper/2101.11517/) (2021)
- [MT3: Meta Test-Time Training for Self-Supervised Test-Time Adaption](https://mlsanity.com/paper/2103.16201/) (2021)
- [Unbiased Gradient Estimation in Unrolled Computation Graphs with Persistent Evolution Strategies](https://mlsanity.com/paper/2112.13835/) (2021)
- [Should We Be Pre-training? An Argument for End-task Aware Training as an Alternative](https://mlsanity.com/paper/2109.07437/) (2021)
- [Noether Networks: Meta-Learning Useful Conserved Quantities](https://mlsanity.com/paper/2112.03321/) (2021)
- [How to Train Your MAML to Excel in Few-Shot Classification](https://mlsanity.com/paper/2106.16245/) (2021)
- [ViT3: Unlocking Test-Time Training in Vision](https://mlsanity.com/paper/2512.01643/) (2025)
- [Fine-grained Recognition with Learnable Semantic Data Augmentation](https://mlsanity.com/paper/2309.00399/) (2023)
- [Learning to (Learn at Test Time)](https://mlsanity.com/paper/2310.13807/) (2023)
- [Few-Shot Learning Meets Transformer: Unified Query-Support Transformers for Few-Shot Classification](https://mlsanity.com/paper/2208.12398/) (2022)
- [A contrastive rule for meta-learning](https://mlsanity.com/paper/2104.01677/) (2021)
- [Meta-Learning Biologically Plausible Plasticity Rules with Random Feedback Pathways](https://mlsanity.com/paper/2210.16414/) (2022)
- [Can Learned Optimization Make Reinforcement Learning Less Difficult?](https://mlsanity.com/paper/2407.07082/) (2024)
- [Test-Time Training with KV Binding Is Secretly Linear Attention](https://mlsanity.com/paper/2602.21204/) (2026)
- [Spiking Neural Networks and Bio-Inspired Supervised Deep Learning: A Survey](https://mlsanity.com/paper/2307.16235/) (2023)
- [Low-Variance Gradient Estimation in Unrolled Computation Graphs with ES-Single](https://mlsanity.com/paper/2304.11153/) (2023)
- [Fast On-Device Adaptation for Spiking Neural Networks via Online-Within-Online Meta-Learning](https://mlsanity.com/paper/2103.03901/) (2021)
- [Contextualizing Meta-Learning via Learning to Decompose](https://mlsanity.com/paper/2106.08112/) (2021)
- [How Should We Meta-Learn Reinforcement Learning Algorithms?](https://mlsanity.com/paper/2507.17668/) (2025)
- [DAC-MR: Data Augmentation Consistency Based Meta-Regularization for Meta-Learning](https://mlsanity.com/paper/2305.07892/) (2023)
- [Learning to Explore for Stochastic Gradient MCMC](https://mlsanity.com/paper/2408.09140/) (2024)
- [Multi-Domain Few-Shot Learning and Dataset for Agricultural Applications](https://mlsanity.com/paper/2109.09952/) (2021)
- [Learning to learn online with neuromodulated synaptic plasticity in spiking neural networks](https://mlsanity.com/paper/2206.12520/) (2022)
- [Federated Automatic Differentiation](https://mlsanity.com/paper/2301.07806/) (2023)
- [Exploiting Field Dependencies for Learning on Categorical Data](https://mlsanity.com/paper/2307.09321/) (2023)
- [Meta-learning representations for clustering with infinite Gaussian mixture models](https://mlsanity.com/paper/2103.00694/) (2021)
- [Meta-learning an Intermediate Representation for Few-shot Block-wise Prediction of Landslide Susceptibility](https://mlsanity.com/paper/2110.04922/) (2021)
- [Compensating trajectory bias for unsupervised patient stratification using adversarial recurrent neural networks](https://mlsanity.com/paper/2112.07239/) (2021)
- [An Adaptive Approach for Probabilistic Wind Power Forecasting Based on Meta-Learning](https://mlsanity.com/paper/2308.07980/) (2023)

## Abstract (arXiv)

A major goal of unsupervised learning is to discover data representations that are useful for subsequent tasks, without access to supervised labels during training. Typically, this involves minimizing a surrogate objective, such as the negative log likelihood of a generative model, with the hope that representations useful for subsequent tasks will arise as a side effect. In this work, we propose instead to directly target later desired tasks by meta-learning an unsupervised learning rule which leads to representations useful for those tasks. Specifically, we target semi-supervised classification performance, and we meta-learn an algorithm -- an unsupervised weight update rule -- that produces representations useful for this task. Additionally, we constrain our unsupervised update rule to a be a biologically-motivated, neuron-local function, which enables it to generalize to different neural network architectures, datasets, and data modalities. We show that the meta-learned update rule produces useful features and sometimes outperforms existing unsupervised learning techniques. We further show that the meta-learned unsupervised update rule generalizes to train networks with different widths, depths, and nonlinearities. It also generalizes to train on data with randomly permuted input dimensions and even generalizes from image datasets to a text task.