Towards a Systems Theory of Algorithms

Traditionally, numerical algorithms are seen as isolated pieces of code confined to an em in silico existence. However, this perspective is not appropriate for many modern computational approaches in control, learning, or optimization, wherein em in vivo algorithms interact with their environment. Examples of such em open algorithms include various real-time optimization-based control strategies, reinforcement learning, decision-making architectures, online optimization, and many more. Further, even em closed algorithms in learning or optimization are increasingly abstracted in block diagrams with interacting dynamic modules and pipelines. In this opinion paper, we state our vision on a to-be-cultivated em systems theory of algorithms and argue in favor of viewing algorithms as open dynamical systems interacting with other algorithms, physical systems, humans, or databases. Remarkably, the manifold tools developed under the umbrella of systems theory are well suited for addressing a range of challenges in the algorithmic domain. We survey various instances where the principles of algorithmic systems theory are being developed and outline pertinent modeling, analysis, and design challenges.