# Towards a Systems Theory of Algorithms

arXiv: https://arxiv.org/abs/2401.14029 · HTML: https://mlsanity.com/paper/2401.14029/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 95 references that are in the map.
- First posted on arXiv 25 Jan 2024.

## Builds on (most cited first, 30 of 38)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://mlsanity.com/paper/2312.00752/) (2023)
- [An Algorithmic Perspective on Imitation Learning](https://mlsanity.com/paper/1811.06711/) (2018)
- [Achieving Geometric Convergence for Distributed Optimization over Time-Varying Graphs](https://mlsanity.com/paper/1607.03218/) (2016)
- [An optimal control perspective on diffusion-based generative modeling](https://mlsanity.com/paper/2211.01364/) (2022)
- [Decision-Focused Learning: Foundations, State of the Art, Benchmark and Future Opportunities](https://mlsanity.com/paper/2307.13565/) (2023)
- [Theoretical guarantees for sampling and inference in generative models with latent diffusions](https://mlsanity.com/paper/1903.01608/) (2019)
- [A Survey of Contextual Optimization Methods for Decision Making under Uncertainty](https://mlsanity.com/paper/2306.10374/) (2023)
- [Performative Prediction: Past and Future](https://mlsanity.com/paper/2310.16608/) (2023)
- [Time-Varying Convex Optimization: Time-Structured Algorithms and Applications](https://mlsanity.com/paper/2006.08500/) (2020)
- [Deep Learning Theory Review: An Optimal Control and Dynamical Systems Perspective](https://mlsanity.com/paper/1908.10920/) (2019)
- [Optimization with Momentum: Dynamical, Control-Theoretic, and Symplectic Perspectives](https://mlsanity.com/paper/2002.12493/) (2020)
- [Data-Driven Model Predictive Control with Stability and Robustness Guarantees](https://mlsanity.com/paper/1906.04679/) (2019)
- [A Classification of Feedback Loops and Their Relation to Biases in Automated Decision-Making Systems](https://mlsanity.com/paper/2305.06055/) (2023)
- [Optimization Algorithms as Robust Feedback Controllers](https://mlsanity.com/paper/2103.11329/) (2021)
- [Analysis and Design of First-Order Distributed Optimization Algorithms over Time-Varying Graphs](https://mlsanity.com/paper/1907.05448/) (2019)
- [Continuous-time Lower Bounds for Gradient-based Algorithms](https://mlsanity.com/paper/2002.03546/) (2020)
- [The Analysis of Optimization Algorithms, A Dissipativity Approach](https://mlsanity.com/paper/2205.14264/) (2022)
- [BIG Hype: Best Intervention in Games via Distributed Hypergradient Descent](https://mlsanity.com/paper/2303.01101/) (2023)
- [On Fundamental Proof Structures in First-Order Optimization](https://mlsanity.com/paper/2310.02015/) (2023)
- [Optimal Power Flow Pursuit](https://mlsanity.com/paper/1601.07263/) (2016)
- [Distributed generalized Nash equilibrium seeking in aggregative games on time-varying networks](https://mlsanity.com/paper/1907.00191/) (2019)
- [An automatic system to detect equivalence between iterative algorithms](https://mlsanity.com/paper/2105.04684/) (2021)
- [Optimization Algorithm Synthesis based on Integral Quadratic Constraints: A Tutorial](https://mlsanity.com/paper/2306.00565/) (2023)
- [InTune: Reinforcement Learning-based Data Pipeline Optimization for Deep Recommendation Models](https://mlsanity.com/paper/2308.08500/) (2023)
- [The Impact of Recommendation Systems on Opinion Dynamics: Microscopic versus Macroscopic Effects](https://mlsanity.com/paper/2309.08967/) (2023)
- [Interpolation Constraints for Computing Worst-Case Bounds in Performance Estimation Problems](https://mlsanity.com/paper/2309.12173/) (2023)
- [Augmented Lagrangian Methods as Layered Control Architectures](https://mlsanity.com/paper/2311.06404/) (2023)
- [The Role of Identification in Data-driven Policy Iteration: A System Theoretic Study](https://mlsanity.com/paper/2401.06721/) (2024)
- [A System Theoretical Perspective to Gradient-Tracking Algorithms for Distributed Quadratic Optimization](https://mlsanity.com/paper/1911.06665/) (2019)

## Built on by (most cited first, 7 of 7)

- [Understanding the differences in Foundation Models: Attention, State Space Models, and Recurrent Neural Networks](https://mlsanity.com/paper/2405.15731/) (2024)
- [Learning to optimize with convergence guarantees using nonlinear system theory](https://mlsanity.com/paper/2403.09389/) (2024)
- [Convergence of energy-based learning in linear resistive networks](https://mlsanity.com/paper/2503.00349/) (2025)
- [Rethinking Optimization: A Systems-Based Approach to Social Externalities](https://mlsanity.com/paper/2506.12825/) (2025)
- [Controlling Participation in Federated Learning with Feedback](https://mlsanity.com/paper/2411.19242/) (2024)
- [Convergence and stability of Q-learning in Hierarchical Reinforcement Learning](https://mlsanity.com/paper/2511.17351/) (2025)
- [A Systems-Theoretic View on the Convergence of Algorithms under Disturbances](https://mlsanity.com/paper/2512.17598/) (2025)

## Abstract (arXiv)

Traditionally, numerical algorithms are seen as isolated pieces of code confined to an em in silico existence. However, this perspective is not appropriate for many modern computational approaches in control, learning, or optimization, wherein em in vivo algorithms interact with their environment. Examples of such em open algorithms include various real-time optimization-based control strategies, reinforcement learning, decision-making architectures, online optimization, and many more. Further, even em closed algorithms in learning or optimization are increasingly abstracted in block diagrams with interacting dynamic modules and pipelines. In this opinion paper, we state our vision on a to-be-cultivated em systems theory of algorithms and argue in favor of viewing algorithms as open dynamical systems interacting with other algorithms, physical systems, humans, or databases. Remarkably, the manifold tools developed under the umbrella of systems theory are well suited for addressing a range of challenges in the algorithmic domain. We survey various instances where the principles of algorithmic systems theory are being developed and outline pertinent modeling, analysis, and design challenges.