# Tailoring: encoding inductive biases by optimizing unsupervised objectives at prediction time

arXiv: https://arxiv.org/abs/2009.10623 · HTML: https://mlsanity.com/paper/2009.10623/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Sep 2020.

## Built on by (most cited first, 13 of 13)

- [MEMO: Test Time Robustness via Adaptation and Augmentation](https://mlsanity.com/paper/2110.09506/) (2021)
- [A Comprehensive Survey on Test-Time Adaptation under Distribution Shifts](https://mlsanity.com/paper/2303.15361/) (2023)
- [Evaluating the Adversarial Robustness of Adaptive Test-time Defenses](https://mlsanity.com/paper/2202.13711/) (2022)
- [TeST: Test-time Self-Training under Distribution Shift](https://mlsanity.com/paper/2209.11459/) (2022)
- [Noether Networks: Meta-Learning Useful Conserved Quantities](https://mlsanity.com/paper/2112.03321/) (2021)
- [OCD: Learning to Overfit with Conditional Diffusion Models](https://mlsanity.com/paper/2210.00471/) (2022)
- [Invariance Through Latent Alignment](https://mlsanity.com/paper/2112.08526/) (2021)
- [Deep Grey-Box Modeling With Adaptive Data-Driven Models Toward Trustworthy Estimation of Theory-Driven Models](https://mlsanity.com/paper/2210.13103/) (2022)
- [Memory-Based Optimization Methods for Model-Agnostic Meta-Learning and Personalized Federated Learning](https://mlsanity.com/paper/2106.04911/) (2021)
- [Probabilistic Test-Time Generalization by Variational Neighbor-Labeling](https://mlsanity.com/paper/2307.04033/) (2023)
- [Deformable Surface Reconstruction via Riemannian Metric Preservation](https://mlsanity.com/paper/2212.11596/) (2022)
- [Functional Risk Minimization](https://mlsanity.com/paper/2412.21149/) (2024)
- [Test Time Adaptation Methods for Point Cloud Registration in Laparoscopic Surgery](https://mlsanity.com/paper/2608.02883/) (2026)

## Abstract (arXiv)

From CNNs to attention mechanisms, encoding inductive biases into neural networks has been a fruitful source of improvement in machine learning. Adding auxiliary losses to the main objective function is a general way of encoding biases that can help networks learn better representations. However, since auxiliary losses are minimized only on training data, they suffer from the same generalization gap as regular task losses. Moreover, by adding a term to the loss function, the model optimizes a different objective than the one we care about. In this work we address both problems: first, we take inspiration from textittransductive learning and note that after receiving an input but before making a prediction, we can fine-tune our networks on any unsupervised loss. We call this process em tailoring, because we customize the model to each input to ensure our prediction satisfies the inductive bias. Second, we formulate em meta-tailoring, a nested optimization similar to that in meta-learning, and train our models to perform well on the task objective after adapting them using an unsupervised loss. The advantages of tailoring and meta-tailoring are discussed theoretically and demonstrated empirically on a diverse set of examples.