# The Alpha-Alternator: Dynamic Adaptation To Varying Noise Levels In Sequences Using The Vendi Score For Improved Robustness and Performance

arXiv: https://arxiv.org/abs/2502.04593 · HTML: https://mlsanity.com/paper/2502.04593/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in Core SSM architectures for language, one of 10 directions in State-space sequence models (S4, Mamba).
- Builds on 20 of its 30 references that are in the map.
- First posted on arXiv 7 Feb 2025.

## Placed in

- State-space sequence models (S4, Mamba) / [Core SSM architectures for language](https://mlsanity.com/ssm/core-ssm-architectures-for-language/)

## Builds on (most cited first, 20 of 20)

- [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://mlsanity.com/paper/2312.00752/) (2023)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [A Recurrent Latent Variable Model for Sequential Data](https://mlsanity.com/paper/1506.02216/) (2015)
- [The Vendi Score: A Diversity Evaluation Metric for Machine Learning](https://mlsanity.com/paper/2210.02410/) (2022)
- [Sequential Neural Models with Stochastic Layers](https://mlsanity.com/paper/1605.07571/) (2016)
- [Is Mamba Effective for Time Series Forecasting?](https://mlsanity.com/paper/2403.11144/) (2024)
- [Diffusion Models for Time Series Applications: A Survey](https://mlsanity.com/paper/2305.00624/) (2023)
- [Deep AutoRegressive Networks](https://mlsanity.com/paper/1310.8499/) (2013)
- [Mamba in Speech: Towards an Alternative to Self-Attention](https://mlsanity.com/paper/2405.12609/) (2024)
- [Neural Diffusion Processes](https://mlsanity.com/paper/2206.03992/) (2022)
- [Machine learning for neural decoding](https://mlsanity.com/paper/1708.00909/) (2017)
- [Beyond Aesthetics: Cultural Competence in Text-to-Image Models](https://mlsanity.com/paper/2407.06863/) (2024)
- [Cousins Of The Vendi Score: A Family Of Similarity-Based Diversity Metrics For Science And Machine Learning](https://mlsanity.com/paper/2310.12952/) (2023)
- [Improving Geo-diversity of Generated Images with Contextualized Vendi Score Guidance](https://mlsanity.com/paper/2406.04551/) (2024)
- [Quality-Weighted Vendi Scores And Their Application To Diverse Experimental Design](https://mlsanity.com/paper/2405.02449/) (2024)
- [Predicting human decision making in psychological tasks with recurrent neural networks](https://mlsanity.com/paper/2010.11413/) (2020)
- [A guide to state-space modeling of ecological time series](https://mlsanity.com/paper/2002.02001/) (2020)
- [Towards Mode Balancing of Generative Models via Diversity Weights](https://mlsanity.com/paper/2304.11961/) (2023)
- [Protein-Mamba: Biological Mamba Models for Protein Function Prediction](https://mlsanity.com/paper/2409.14617/) (2024)
- [Alternators For Sequence Modeling](https://mlsanity.com/paper/2405.11848/) (2024)

## Built on by (most cited first, 5 of 5)

- [Vendi-RAG: Adaptively Trading-Off Diversity And Quality Significantly Improves Retrieval Augmented Generation With LLMs](https://mlsanity.com/paper/2502.11228/) (2025)
- [The Vendiscope: An Algorithmic Microscope For Data Collections](https://mlsanity.com/paper/2502.10828/) (2025)
- [Vendi Novelty Scores for Out-of-Distribution Detection](https://mlsanity.com/paper/2602.10062/) (2026)
- [Alternators With Noise Models](https://mlsanity.com/paper/2505.12544/) (2025)
- [Vendi Information Gain for Active Learning and its Application to Ecology](https://mlsanity.com/paper/2509.10390/) (2025)

## Abstract (arXiv)

Current state-of-the-art dynamical models, such as Mamba, assume the same level of noisiness for all elements of a given sequence, which limits their performance on noisy temporal data. In this paper, we introduce the α-Alternator, a novel generative model for time-dependent data that dynamically adapts to the complexity introduced by varying noise levels in sequences. The α-Alternator leverages the Vendi Score (VS), a flexible similarity-based diversity metric, to adjust, at each time step t, the influence of the sequence element at time t and the latent representation of the dynamics up to that time step on the predicted future dynamics. This influence is captured by a parameter that is learned and shared across all sequences in a given dataset. The sign of this parameter determines the direction of influence. A negative value indicates a noisy dataset, where a sequence element that increases the VS is considered noisy, and the model relies more on the latent history when processing that element. Conversely, when the parameter is positive, a sequence element that increases the VS is considered informative, and the α-Alternator relies more on this new input than on the latent history when updating its predicted latent dynamics. The α-Alternator is trained using a combination of observation masking and Alternator loss minimization. Masking simulates varying noise levels in sequences, enabling the model to be more robust to these fluctuations and improving its performance in trajectory prediction, imputation, and forecasting. Our experimental results demonstrate that the α-Alternator outperforms both Alternators and state-of-the-art state-space models across neural decoding and time-series forecasting benchmarks.