# Geometric SSM: LTI State Space Models for Selective Tasks

arXiv: https://arxiv.org/abs/2505.17932 · HTML: https://mlsanity.com/paper/2505.17932/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Core SSM architectures for language, one of 10 directions in State-space sequence models (S4, Mamba).
- Builds on 19 of its 25 references that are in the map.
- First posted on arXiv 23 May 2025.

## Placed in

- State-space sequence models (S4, Mamba) / [Core SSM architectures for language](https://mlsanity.com/ssm/core-ssm-architectures-for-language/)

## Builds on (most cited first, 19 of 19)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://mlsanity.com/paper/2312.00752/) (2023)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [Efficiently Modeling Long Sequences with Structured State Spaces](https://mlsanity.com/paper/2111.00396/) (2021)
- [Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention](https://mlsanity.com/paper/2006.16236/) (2020)
- [Rethinking Attention with Performers](https://mlsanity.com/paper/2009.14794/) (2020)
- [Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality](https://mlsanity.com/paper/2405.21060/) (2024)
- [Efficient Transformers: A Survey](https://mlsanity.com/paper/2009.06732/) (2020)
- [Simplified State Space Layers for Sequence Modeling](https://mlsanity.com/paper/2208.04933/) (2022)
- [Hungry Hungry Hippos: Towards Language Modeling with State Space Models](https://mlsanity.com/paper/2212.14052/) (2022)
- [Activation Functions in Deep Learning: A Comprehensive Survey and Benchmark](https://mlsanity.com/paper/2109.14545/) (2021)
- [Liquid Structural State-Space Models](https://mlsanity.com/paper/2209.12951/) (2022)
- [Theoretical Foundations of Deep Selective State-Space Models](https://mlsanity.com/paper/2402.19047/) (2024)
- [Induction Heads as an Essential Mechanism for Pattern Matching in In-context Learning](https://mlsanity.com/paper/2407.07011/) (2024)
- [B'MOJO: Hybrid State Space Realizations of Foundation Models with Eidetic and Fading Memory](https://mlsanity.com/paper/2407.06324/) (2024)
- [Parallelizing Legendre Memory Unit Training](https://mlsanity.com/paper/2102.11417/) (2021)
- [State-Free Inference of State-Space Models: The Transfer Function Approach](https://mlsanity.com/paper/2405.06147/) (2024)

## Abstract (arXiv)

A key claim in recent work on Selective State Space Models is that selectivity, the ability to focus on relevant information while filtering irrelevant inputs, requires breaking the Linear Time-Invariant (LTI) property through time-varying dynamics. We challenge this claim by demonstrating that LTI systems can achieve selectivity when designed using principles from geometric control. We introduce the Geometric SSM, in which different input patterns excite distinct invariant subspaces of the dynamics. Unlike Mamba's memoryless selection mechanism, our approach employs a dynamic residual generator that maintains temporal memory, enabling recognition of multi-token patterns without time-varying system matrices. The Geometric SSM achieves near-perfect performance on a novel extended induction head task where Mamba fails, while preserving efficient FFT-based training. Our results demonstrate that geometric control theory can inform the design of novel selective sequence models that combine theoretical rigor with practical efficiency.