# Adaptive model selection in photonic reservoir computing by reinforcement learning

arXiv: https://arxiv.org/abs/2004.12575 · HTML: https://mlsanity.com/paper/2004.12575/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Apr 2020.

## Built on by (most cited first, 6 of 6)

- [Training of Physical Neural Networks](https://mlsanity.com/paper/2406.03372/) (2024)
- [Attention-Enhanced Reservoir Computing](https://mlsanity.com/paper/2312.16503/) (2023)
- [Theory of Acceleration of Decision Making by Correlated Time Sequences](https://mlsanity.com/paper/2203.16004/) (2022)
- [Building Intelligence in the Mechanical Domain -- Harvesting the Reservoir Computing Power in Origami to Achieve Information Perception Tasks](https://mlsanity.com/paper/2302.05517/) (2023)
- [Model-free Optical Processors using In Situ Reinforcement Learning with Proximal Policy Optimization](https://mlsanity.com/paper/2507.05583/) (2025)
- [Autocorrelation effects in a stochastic-process model for solving two-armed bandit problems](https://mlsanity.com/paper/2603.05559/) (2026)

## Abstract (arXiv)

Photonic reservoir computing is an emergent technology toward beyond-Neumann computing. Although photonic reservoir computing provides superior performance in environments whose characteristics are coincident with the training datasets for the reservoir, the performance is significantly degraded if these characteristics deviate from the original knowledge used in the training phase. Here, we propose a scheme of adaptive model selection in photonic reservoir computing using reinforcement learning. In this scheme, a temporal waveform is generated by different dynamic source models that change over time. The system autonomously identifies the best source model for the task of time series prediction using photonic reservoir computing and reinforcement learning. We prepare two types of output weights for the source models, and the system adaptively selected the correct model using reinforcement learning, where the prediction errors are associated with rewards. We succeed in adaptive model selection when the source signal is temporally mixed, having originally been generated by two different dynamic system models, as well as when the signal is a mixture from the same model but with different parameter values. This study paves the way for autonomous behavior in photonic artificial intelligence and could lead to new applications in load forecasting and multi-objective control, where frequent environment changes are expected.