# A Deep State-Space Model Compression Method using Upper Bound on Output Error

arXiv: https://arxiv.org/abs/2510.14542 · HTML: https://mlsanity.com/paper/2510.14542/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 23 of its 30 references that are in the map.
- First posted on arXiv 16 Oct 2025.

## Builds on (most cited first, 23 of 23)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://mlsanity.com/paper/2312.00752/) (2023)
- [Pointer Sentinel Mixture Models](https://mlsanity.com/paper/1609.07843/) (2016)
- [Efficiently Modeling Long Sequences with Structured State Spaces](https://mlsanity.com/paper/2111.00396/) (2021)
- [Simplified State Space Layers for Sequence Modeling](https://mlsanity.com/paper/2208.04933/) (2022)
- [Long Range Arena: A Benchmark for Efficient Transformers](https://mlsanity.com/paper/2011.04006/) (2020)
- [Combining Recurrent, Convolutional, and Continuous-time Models with Linear State-Space Layers](https://mlsanity.com/paper/2110.13985/) (2021)
- [HiPPO: Recurrent Memory with Optimal Polynomial Projections](https://mlsanity.com/paper/2008.07669/) (2020)
- [On the Parameterization and Initialization of Diagonal State Space Models](https://mlsanity.com/paper/2206.11893/) (2022)
- [State Space Model for New-Generation Network Alternative to Transformers: A Survey](https://mlsanity.com/paper/2404.09516/) (2024)
- [Identification of block-oriented nonlinear systems starting from linear approximations: A survey](https://mlsanity.com/paper/1607.01217/) (2016)
- [Layer-Adaptive State Pruning for Deep State Space Models](https://mlsanity.com/paper/2411.02824/) (2024)
- [Model Compression Method for S4 with Diagonal State Space Layers using Balanced Truncation](https://mlsanity.com/paper/2402.15993/) (2024)
- [Model order reduction of deep structured state-space models: A system-theoretic approach](https://mlsanity.com/paper/2403.14833/) (2024)
- [State Space Models are Provably Comparable to Transformers in Dynamic Token Selection](https://mlsanity.com/paper/2405.19036/) (2024)
- [The Curious Case of In-Training Compression of State Space Models](https://mlsanity.com/paper/2510.02823/) (2025)
- [Compression Method for Deep Diagonal State Space Model Based on H2 Optimal Reduction](https://mlsanity.com/paper/2507.10078/) (2025)
- [Gramians, Energy Functionals and Balanced Truncation for Linear Dynamical Systems with Quadratic Outputs](https://mlsanity.com/paper/1909.04597/) (2019)
- [Data-driven modeling of linear dynamical systems with quadratic output in the AAA framework](https://mlsanity.com/paper/2102.13487/) (2021)
- [Balanced Truncation of Linear Systems with Quadratic Outputs in Limited Time and Frequency Intervals](https://mlsanity.com/paper/2402.11445/) (2024)
- [H2 optimal model reduction of linear systems with multiple quadratic outputs](https://mlsanity.com/paper/2405.05951/) (2024)
- [Time-limited H2-optimal Model Order Reduction of Linear Systems with Quadratic Outputs](https://mlsanity.com/paper/2408.05965/) (2024)
- [H2-optimal model reduction of linear quadratic-output systems by multivariate rational interpolation](https://mlsanity.com/paper/2505.03057/) (2025)

## Abstract (arXiv)

We study deep state-space models (Deep SSMs) that contain linear quadratic-output (LQO) systems as internal blocks and present a compression method with a provable output error guarantee. We first derive an upper bound on the output error between two Deep SSMs and show that the bound can be expressed in terms of the h2-error norms between the layerwise LQO systems. In particular, we show that reducing the h2 approximation errors of the LQO systems placed in shallow layers is effective in reducing the derived upper bound on the output error. Next, we formulate an optimization problem for the derived upper bound and develop a gradient-based MOR method. In the numerical experiments, using the IMDb task from the LRA benchmark, we demonstrate the effectiveness of the proposed upper-bound-based compression method. In particular, we show that the number of trainable parameters can be reduced by approximately 60% without retraining while maintaining the performance of the original model.