Reinforcement Learning-Based Output Feedback LQR for Continuous-Time MIMO Systems

This article studies model-free output feedback linear quadratic regulation (LQR) for continuous-time linear systems with an n-dimensional state, an m-dimensional input, and a p-dimensional output, using filtered input--output data. Since the system state is unavailable, existing methods rely on dynamic filters to parameterize the hidden state using measurable input--output signals. However, the intrinsic dimension of the resulting filter-based parametrization can be smaller than the dimension of the complete filtered vector, and this deterministic redundancy can make the Bellman regressions rank deficient. We characterize this intrinsic dimension and show that the conventional filtered vector contains only 2n independent components for single-input multi-output (SIMO) systems and n(m+1) independent components for general multi-input multi-output (MIMO) systems. Based on this characterization, a reduced filtered vector is extracted directly from data and used to develop reduced model-free output feedback policy iteration and value iteration equations, eliminating the redundant directions and decreasing the number of unknown parameters while retaining a fully input--output data-based implementation. A numerical example illustrates the rank reduction and the effectiveness of the learned controller.