# Information-Theoretic Bounds on Transfer Generalization Gap Based on Jensen-Shannon Divergence

arXiv: https://arxiv.org/abs/2010.09484 · HTML: https://mlsanity.com/paper/2010.09484/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Oct 2020.

## Built on by (most cited first, 11 of 11)

- [Few-Shot Calibration of Set Predictors via Meta-Learned Cross-Validation-Based Conformal Prediction](https://mlsanity.com/paper/2210.03067/) (2022)
- [Generalization Bounds for Noisy Iterative Algorithms Using Properties of Additive Noise Channels](https://mlsanity.com/paper/2102.02976/) (2021)
- [Information-Theoretic Analysis of Unsupervised Domain Adaptation](https://mlsanity.com/paper/2210.00706/) (2022)
- [Information-Theoretic Characterization of the Generalization Error for Iterative Semi-Supervised Learning](https://mlsanity.com/paper/2110.00926/) (2021)
- [An Information-Theoretic Analysis of the Impact of Task Similarity on Meta-Learning](https://mlsanity.com/paper/2101.08390/) (2021)
- [Transfer Learning for Quantum Classifiers: An Information-Theoretic Generalization Analysis](https://mlsanity.com/paper/2201.06297/) (2022)
- [Information-Theoretic Generalization Bounds for Transductive Learning and its Applications](https://mlsanity.com/paper/2311.04561/) (2023)
- [Quantifying Distribution Shifts and Uncertainties for Enhanced Model Robustness in Machine Learning Applications](https://mlsanity.com/paper/2405.01978/) (2024)
- [ReSpec: Relevance and Specificity Grounded Online Filtering for Learning on Video-Text Data Streams](https://mlsanity.com/paper/2504.14875/) (2025)
- [Transfer Bayesian Meta-learning via Weighted Free Energy Minimization](https://mlsanity.com/paper/2106.10711/) (2021)
- [StegaFFD: Privacy-Preserving Face Forgery Detection via Fine-Grained Steganographic Domain Lifting](https://mlsanity.com/paper/2603.02886/) (2026)

## Abstract (arXiv)

In transfer learning, training and testing data sets are drawn from different data distributions. The transfer generalization gap is the difference between the population loss on the target data distribution and the training loss. The training data set generally includes data drawn from both source and target distributions. This work presents novel information-theoretic upper bounds on the average transfer generalization gap that capture (i) the domain shift between the target data distribution P'Z and the source distribution PZ through a two-parameter family of generalized (α1,α2)-Jensen-Shannon (JS) divergences; and (ii) the sensitivity of the transfer learner output W to each individual sample of the data set Zi via the mutual information I(W;Zi). For α1 ∈ (0,1), the (α1,α2)-JS divergence can be bounded even when the support of PZ is not included in that of P'Z. This contrasts the Kullback-Leibler (KL) divergence DKL(PZ||P'Z)-based bounds of Wu et al. [1], which are vacuous under this assumption. Moreover, the obtained bounds hold for unbounded loss functions with bounded cumulant generating functions, unlike the φ-divergence based bound of Wu et al. [1]. We also obtain new upper bounds on the average transfer excess risk in terms of the (α1,α2)-JS divergence for empirical weighted risk minimization (EWRM), which minimizes the weighted average training losses over source and target data sets. Finally, we provide a numerical example to illustrate the merits of the introduced bounds.