# Feature-Model-Guided Online Learning for Self-Adaptive Systems

arXiv: https://arxiv.org/abs/1907.09158 · HTML: https://mlsanity.com/paper/1907.09158/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Jul 2019.

## Built on by (most cited first, 7 of 7)

- [Explaining Online Reinforcement Learning Decisions of Self-Adaptive Systems](https://mlsanity.com/paper/2210.05931/) (2022)
- [A User Study on Explainable Online Reinforcement Learning for Adaptive Systems](https://mlsanity.com/paper/2307.04098/) (2023)
- [Deep Learning for Effective and Efficient Reduction of Large Adaptation Spaces in Self-Adaptive Systems](https://mlsanity.com/paper/2204.06254/) (2022)
- [An AI Chatbot for Explaining Deep Reinforcement Learning Decisions of Service-oriented Systems](https://mlsanity.com/paper/2309.14391/) (2023)
- [Autonomous Learning From Success and Failure: Goal-Conditioned Supervised Learning with Negative Feedback](https://mlsanity.com/paper/2509.03206/) (2025)
- [POLARIS: Is Multi-Agentic Reasoning the Next Wave in Engineering Self-Adaptive Systems?](https://mlsanity.com/paper/2512.04702/) (2025)
- [TwinLoop: Simulation-in-the-Loop Digital Twins for Online Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2604.06610/) (2026)

## Abstract (arXiv)

A self-adaptive system can modify its own structure and behavior at runtime based on its perception of the environment, of itself and of its requirements. To develop a self-adaptive system, software developers codify knowledge about the system and its environment, as well as how adaptation actions impact on the system. However, the codified knowledge may be insufficient due to design time uncertainty, and thus a self-adaptive system may execute adaptation actions that do not have the desired effect. Online learning is an emerging approach to address design time uncertainty by employing machine learning at runtime. Online learning accumulates knowledge at runtime by, for instance, exploring not-yet executed adaptation actions. We address two specific problems with respect to online learning for self-adaptive systems. First, the number of possible adaptation actions can be very large. Existing online learning techniques randomly explore the possible adaptation actions, but this can lead to slow convergence of the learning process. Second, the possible adaptation actions can change as a result of system evolution. Existing online learning techniques are unaware of these changes and thus do not explore new adaptation actions, but explore adaptation actions that are no longer valid. We propose using feature models to give structure to the set of adaptation actions and thereby guide the exploration process during online learning. Experimental results involving four real-world systems suggest that considering the hierarchical structure of feature models may speed up convergence by 7.2% on average. Considering the differences between feature models before and after an evolution step may speed up convergence by 64.6% on average. [...]