# Fast Context Adaptation in Cost-Aware Continual Learning

arXiv: https://arxiv.org/abs/2306.03887 · HTML: https://mlsanity.com/paper/2306.03887/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 43 references that are in the map.
- First posted on arXiv 6 Jun 2023.

## Builds on (most cited first, 27 of 27)

- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [Learning without Forgetting](https://mlsanity.com/paper/1606.09282/) (2016)
- [Experience Replay for Continual Learning](https://mlsanity.com/paper/1811.11682/) (2018)
- [Meta-Learning in Neural Networks: A Survey](https://mlsanity.com/paper/2004.05439/) (2020)
- [Regularized Evolution for Image Classifier Architecture Search](https://mlsanity.com/paper/1802.01548/) (2018)
- [PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruning](https://mlsanity.com/paper/1711.05769/) (2017)
- [Learning to learn by gradient descent by gradient descent](https://mlsanity.com/paper/1606.04474/) (2016)
- [Progress & Compress: A scalable framework for continual learning](https://mlsanity.com/paper/1805.06370/) (2018)
- [How to train your MAML](https://mlsanity.com/paper/1810.09502/) (2018)
- [Learn to Grow: A Continual Structure Learning Framework for Overcoming Catastrophic Forgetting](https://mlsanity.com/paper/1904.00310/) (2019)
- [Learning to Adapt in Dynamic, Real-World Environments Through Meta-Reinforcement Learning](https://mlsanity.com/paper/1803.11347/) (2018)
- [Selective Experience Replay for Lifelong Learning](https://mlsanity.com/paper/1802.10269/) (2018)
- [Reinforced Continual Learning](https://mlsanity.com/paper/1805.12369/) (2018)
- [Applications of Deep Reinforcement Learning in Communications and Networking: A Survey](https://mlsanity.com/paper/1810.07862/) (2018)
- [The Roadmap to 6G -- AI Empowered Wireless Networks](https://mlsanity.com/paper/1904.11686/) (2019)
- [Uncertainty-based Continual Learning with Adaptive Regularization](https://mlsanity.com/paper/1905.11614/) (2019)
- [Online Meta-Learning](https://mlsanity.com/paper/1902.08438/) (2019)
- [Meta-Reinforcement Learning of Structured Exploration Strategies](https://mlsanity.com/paper/1802.07245/) (2018)
- [Learned Optimizers that Scale and Generalize](https://mlsanity.com/paper/1703.04813/) (2017)
- [Evolved Policy Gradients](https://mlsanity.com/paper/1802.04821/) (2018)
- [A Lightweight Concept Drift Detection and Adaptation Framework for IoT Data Streams](https://mlsanity.com/paper/2104.10529/) (2021)
- [Software-Defined and Virtualized Future Mobile and Wireless Networks: A Survey](https://mlsanity.com/paper/1409.0079/) (2014)
- [No Free Lunch: Balancing Learning and Exploitation at the Network Edge](https://mlsanity.com/paper/2111.11912/) (2021)
- [Energy Efficient Deployment and Orchestration of Computing Resources at the Network Edge: a Survey on Algorithms, Trends and Open Challenges](https://mlsanity.com/paper/2209.14141/) (2022)
- [Using Distributed Reinforcement Learning for Resource Orchestration in a Network Slicing Scenario](https://mlsanity.com/paper/2105.07946/) (2021)
- [The Cost of Learning: Efficiency vs. Efficacy of Learning-Based RRM for 6G](https://mlsanity.com/paper/2211.16915/) (2022)

## Built on by (most cited first, 2 of 2)

- [Out-of-Distribution Generalization with a SPARC: Racing 100 Unseen Vehicles with a Single Policy](https://mlsanity.com/paper/2511.09737/) (2025)
- [To Train or Not to Train: Balancing Efficiency and Training Cost in Deep Reinforcement Learning for Mobile Edge Computing](https://mlsanity.com/paper/2411.07086/) (2024)

## Abstract (arXiv)

In the past few years, DRL has become a valuable solution to automatically learn efficient resource management strategies in complex networks with time-varying statistics. However, the increased complexity of 5G and Beyond networks requires correspondingly more complex learning agents and the learning process itself might end up competing with users for communication and computational resources. This creates friction: on the one hand, the learning process needs resources to quickly convergence to an effective strategy; on the other hand, the learning process needs to be efficient, i.e., take as few resources as possible from the user's data plane, so as not to throttle users' QoS. In this paper, we investigate this trade-off and propose a dynamic strategy to balance the resources assigned to the data plane and those reserved for learning. With the proposed approach, a learning agent can quickly converge to an efficient resource allocation strategy and adapt to changes in the environment as for the CL paradigm, while minimizing the impact on the users' QoS. Simulation results show that the proposed method outperforms static allocation methods with minimal learning overhead, almost reaching the performance of an ideal out-of-band CL solution.