# Everything old is new again: A multi-view learning approach to learning using privileged information and distillation

arXiv: https://arxiv.org/abs/1903.03694 · HTML: https://mlsanity.com/paper/1903.03694/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Mar 2019.

## Built on by (most cited first, 2 of 2)

- [Efficient learning of nonlinear prediction models with time-series privileged information](https://mlsanity.com/paper/2209.07067/) (2022)
- [Large Scale Long-tailed Product Recognition System at Alibaba](https://mlsanity.com/paper/2102.04652/) (2021)

## Abstract (arXiv)

We adopt a multi-view approach for analyzing two knowledge transfer settings---learning using privileged information (LUPI) and distillation---in a common framework. Under reasonable assumptions about the complexities of hypothesis spaces, and being optimistic about the expected loss achievable by the student (in distillation) and a transformed teacher predictor (in LUPI), we show that encouraging agreement between the teacher and the student leads to reduced search space. As a result, improved convergence rate can be obtained with regularized empirical risk minimization.