# Tydra: An Efficient Hybrid Model for Tabular Data

arXiv: https://arxiv.org/abs/2608.21199 · HTML: https://mlsanity.com/paper/2608.21199/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 17 references that are in the map.
- First posted on arXiv 21 Aug 2026.

## Builds on (most cited first, 12 of 12)

- [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://mlsanity.com/paper/2312.00752/) (2023)
- [Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality](https://mlsanity.com/paper/2405.21060/) (2024)
- [TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Second](https://mlsanity.com/paper/2207.01848/) (2022)
- [Universal Transformers](https://mlsanity.com/paper/1807.03819/) (2018)
- [Jamba: A Hybrid Transformer-Mamba Language Model](https://mlsanity.com/paper/2403.19887/) (2024)
- [Zamba: A Compact 7B SSM Hybrid Model](https://mlsanity.com/paper/2405.16712/) (2024)
- [Hydra: Bidirectional State Space Models Through Generalized Matrix Mixers](https://mlsanity.com/paper/2407.09941/) (2024)
- [A Closer Look at TabPFN v2: Understanding Its Strengths and Extending Its Capabilities](https://mlsanity.com/paper/2502.17361/) (2025)
- [Olmo Hybrid: From Theory to Practice and Back](https://mlsanity.com/paper/2604.03444/) (2026)
- [Advancing Intelligent Sequence Modeling: Evolution, Trade-offs, and Applications of State-Space Architectures from S4 to Mamba](https://mlsanity.com/paper/2503.18970/) (2025)
- [TransMamba: A Sequence-Level Hybrid Transformer-Mamba Language Model](https://mlsanity.com/paper/2503.24067/) (2025)
- [State-Space Models for Tabular Prior-Data Fitted Networks](https://mlsanity.com/paper/2510.14573/) (2025)

## Built on by (most cited first, 1 of 1)

- [LoopICL: Looping a single transformer block to solve tabular tasks](https://mlsanity.com/paper/2609.36108/) (2026)

## Abstract (arXiv)

Transformer-based tabular foundation models such as TabPFN achieve strong predictive performance but incur quadratic computational cost with context length. On the other hand, subquadratic SSM-based alternatives such as Hydra trade away accuracy for efficiency. To balance both, we introduce Tydra, a hybrid Transformer-State Space Model (SSM) architecture for tabular in-context learning that interleaves attention and SSM layers. Across 30 OpenML datasets, Tydra reduces inference time by 30% relative to TabPFN while retaining much of its predictive performance. Tydra also outperforms an approximately ten-times-larger Hydra model while providing faster inference. The results indicate that hybrid architectures are a promising direction for tabular foundation models.