# Fast Rates by Transferring from Auxiliary Hypotheses

arXiv: https://arxiv.org/abs/1412.1619 · HTML: https://mlsanity.com/paper/1412.1619/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Dec 2014.

## Built on by (most cited first, 16 of 16)

- [Knowledge Distillation with the Reused Teacher Classifier](https://mlsanity.com/paper/2203.14001/) (2022)
- [Characterizing and Understanding the Generalization Error of Transfer Learning with Gibbs Algorithm](https://mlsanity.com/paper/2111.01635/) (2021)
- [Generalized Knowledge Distillation via Relationship Matching](https://mlsanity.com/paper/2205.01915/) (2022)
- [Smoothness Adaptive Hypothesis Transfer Learning](https://mlsanity.com/paper/2402.14966/) (2024)
- [Transfer Learning with Kernel Methods](https://mlsanity.com/paper/2211.00227/) (2022)
- [Gap Minimization for Knowledge Sharing and Transfer](https://mlsanity.com/paper/2201.11231/) (2022)
- [Hypothesis Transfer Learning with Surrogate Classification Losses: Generalization Bounds through Algorithmic Stability](https://mlsanity.com/paper/2305.19694/) (2023)
- [Transfer learning with affine model transformation](https://mlsanity.com/paper/2210.09745/) (2022)
- [On Hypothesis Transfer Learning of Functional Linear Models](https://mlsanity.com/paper/2206.04277/) (2022)
- [Hypothesis Transfer in Bandits by Weighted Models](https://mlsanity.com/paper/2211.07387/) (2022)
- [ZhiJian: A Unifying and Rapidly Deployable Toolbox for Pre-trained Model Reuse](https://mlsanity.com/paper/2308.09158/) (2023)
- [Fixed-Gaussian Spectral Algorithms: Minimax Optimal Rates for Misspecified Learning and Transfer](https://mlsanity.com/paper/2501.10870/) (2025)
- [Mixed-Sample SGD: an End-to-end Analysis of Supervised Transfer Learning](https://mlsanity.com/paper/2507.04194/) (2025)
- [Source-free Domain Adaptation Requires Penalized Diversity](https://mlsanity.com/paper/2304.02798/) (2023)
- [Transfer Learning for High-dimensional Reduced Rank Time Series Models](https://mlsanity.com/paper/2504.15691/) (2025)
- [GeoERM: Geometry-Aware Multi-Task Representation Learning on Riemannian Manifolds](https://mlsanity.com/paper/2505.02972/) (2025)

## Abstract (arXiv)

In this work we consider the learning setting where, in addition to the training set, the learner receives a collection of auxiliary hypotheses originating from other tasks. We focus on a broad class of ERM-based linear algorithms that can be instantiated with any non-negative smooth loss function and any strongly convex regularizer. We establish generalization and excess risk bounds, showing that, if the algorithm is fed with a good combination of source hypotheses, generalization happens at the fast rate O(1/m) instead of the usual O(1/√m). On the other hand, if the source hypotheses combination is a misfit for the target task, we recover the usual learning rate. As a byproduct of our study, we also prove a new bound on the Rademacher complexity of the smooth loss class under weaker assumptions compared to previous works.