Regret Analysis with Almost Sure Convergence for OBF-ARX Filter

This paper considers the output prediction problem for an unknown Linear Time-Invariant (LTI) system. In particular, we focus our attention on the OBF-ARX filter, whose transfer function is a linear combination of Orthogonal Basis Functions (OBFs), with the coefficients determined by solving a least-squares regression. We prove that the OBF-ARX filter is an accurate approximation of the Kalman Filter (KF) by quantifying its online performance. Specifically, we analyze the average regret between the OBF-ARX filter and the KF, proving that the average regret over N time steps converges to the asymptotic bias at the speed of O(N-0.5+ε) almost surely for all ε>0. Then, we establish an upper bound on the asymptotic bias, demonstrating that it decreases exponentially with the number of OBF bases, and the decreasing rate τ(boldsymbolλ, boldsymbolμ) explicitly depends on the poles of both the KF and the OBF. Numerical results on diffusion processes validate the derived bounds.