# Guided Transfer Learning

arXiv: https://arxiv.org/abs/2303.16154 · HTML: https://mlsanity.com/paper/2303.16154/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 26 references that are in the map.
- First posted on arXiv 26 Mar 2023.

## Builds on (most cited first, 14 of 14)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Training Compute-Optimal Large Language Models](https://mlsanity.com/paper/2203.15556/) (2022)
- [Matching Networks for One Shot Learning](https://mlsanity.com/paper/1606.04080/) (2016)
- [The Computational Limits of Deep Learning](https://mlsanity.com/paper/2007.05558/) (2020)
- [Continual Learning of Context-dependent Processing in Neural Networks](https://mlsanity.com/paper/1810.01256/) (2018)
- [Benchmark Analysis of Representative Deep Neural Network Architectures](https://mlsanity.com/paper/1810.00736/) (2018)
- [The Omniglot challenge: a 3-year progress report](https://mlsanity.com/paper/1902.03477/) (2019)
- [Active One-shot Learning](https://mlsanity.com/paper/1702.06559/) (2017)
- [Power-law Scaling to Assist with Key Challenges in Artificial Intelligence](https://mlsanity.com/paper/2211.08430/) (2022)
- [Practopoiesis: Or how life fosters a mind](https://mlsanity.com/paper/1402.5332/) (2014)
- [Where is the mind within the brain? Transient selection of subnetworks by metabotropic receptors and G protein-gated ion channels](https://mlsanity.com/paper/2207.11249/) (2022)

## Built on by (most cited first, 2 of 2)

- [Federated Loss Exploration for Improved Convergence on Non-IID Data](https://mlsanity.com/paper/2506.18640/) (2025)
- [Pruning Federated Models through Loss Landscape Analysis and Client Agreement Scoring](https://mlsanity.com/paper/2405.10271/) (2024)

## Abstract (arXiv)

Machine learning requires exuberant amounts of data and computation. Also, models require equally excessive growth in the number of parameters. It is, therefore, sensible to look for technologies that reduce these demands on resources. Here, we propose an approach called guided transfer learning. Each weight and bias in the network has its own guiding parameter that indicates how much this parameter is allowed to change while learning a new task. Guiding parameters are learned during an initial scouting process. Guided transfer learning can result in a reduction in resources needed to train a network. In some applications, guided transfer learning enables the network to learn from a small amount of data. In other cases, a network with a smaller number of parameters can learn a task which otherwise only a larger network could learn. Guided transfer learning potentially has many applications when the amount of data, model size, or the availability of computational resources reach their limits.