# Benchmarking M-LTSF: Frequency and Noise-Based Evaluation of Multivariate Long Time Series Forecasting Models

arXiv: https://arxiv.org/abs/2510.04900 · HTML: https://mlsanity.com/paper/2510.04900/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Time series forecasting with SSMs, one of 10 directions in State-space sequence models (S4, Mamba).
- Builds on 29 of its 48 references that are in the map.
- First posted on arXiv 6 Oct 2025.

## Placed in

- State-space sequence models (S4, Mamba) / [Time series forecasting with SSMs](https://mlsanity.com/ssm/time-series-forecasting-with-ssms/)

## Builds on (most cited first, 29 of 29)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://mlsanity.com/paper/2312.00752/) (2023)
- [Efficiently Modeling Long Sequences with Structured State Spaces](https://mlsanity.com/paper/2111.00396/) (2021)
- [Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting](https://mlsanity.com/paper/2012.07436/) (2020)
- [Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting](https://mlsanity.com/paper/2106.13008/) (2021)
- [Are Transformers Effective for Time Series Forecasting?](https://mlsanity.com/paper/2205.13504/) (2022)
- [FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecasting](https://mlsanity.com/paper/2201.12740/) (2022)
- [iTransformer: Inverted Transformers Are Effective for Time Series Forecasting](https://mlsanity.com/paper/2310.06625/) (2023)
- [Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networks](https://mlsanity.com/paper/1703.07015/) (2017)
- [Temporal Fusion Transformers for Interpretable Multi-horizon Time Series Forecasting](https://mlsanity.com/paper/1912.09363/) (2019)
- [Chronos: Learning the Language of Time Series](https://mlsanity.com/paper/2403.07815/) (2024)
- [A decoder-only foundation model for time-series forecasting](https://mlsanity.com/paper/2310.10688/) (2023)
- [Unified Training of Universal Time Series Forecasting Transformers](https://mlsanity.com/paper/2402.02592/) (2024)
- [Non-stationary Transformers: Exploring the Stationarity in Time Series Forecasting](https://mlsanity.com/paper/2205.14415/) (2022)
- [TFB: Towards Comprehensive and Fair Benchmarking of Time Series Forecasting Methods](https://mlsanity.com/paper/2403.20150/) (2024)
- [GIFT-Eval: A Benchmark For General Time Series Forecasting Model Evaluation](https://mlsanity.com/paper/2410.10393/) (2024)
- [LargeST: A Benchmark Dataset for Large-Scale Traffic Forecasting](https://mlsanity.com/paper/2306.08259/) (2023)
- [Is Mamba Effective for Time Series Forecasting?](https://mlsanity.com/paper/2403.11144/) (2024)
- [A Review Paper: Noise Models in Digital Image Processing](https://mlsanity.com/paper/1505.03489/) (2015)
- [Deep Learning for Road Traffic Forecasting: Does it Make a Difference?](https://mlsanity.com/paper/2012.02260/) (2020)
- [Robust Probabilistic Time Series Forecasting](https://mlsanity.com/paper/2202.11910/) (2022)
- [Dynamical errors in machine learning forecasts](https://mlsanity.com/paper/2504.11074/) (2025)
- [HR-Extreme: A High-Resolution Dataset for Extreme Weather Forecasting](https://mlsanity.com/paper/2409.18885/) (2024)
- [Climate Aware Deep Neural Networks (CADNN) for Wind Power Simulation](https://mlsanity.com/paper/2412.12160/) (2024)
- [Synthetic Time Series Forecasting with Transformer Architectures: Extensive Simulation Benchmarks](https://mlsanity.com/paper/2505.20048/) (2025)
- [Stecformer: Spatio-temporal Encoding Cascaded Transformer for Multivariate Long-term Time Series Forecasting](https://mlsanity.com/paper/2305.16370/) (2023)
- [HADL Framework for Noise Resilient Long-Term Time Series Forecasting](https://mlsanity.com/paper/2502.10569/) (2025)
- [S4D-Bio Audio Monitoring of Bone Cement Disintegration in Pulsating Fluid Jet Surgery under Laboratory Conditions](https://mlsanity.com/paper/2503.02714/) (2025)

## Built on by (most cited first, 1 of 1)

- [Benchmarking Sensor-Fault Robustness in Forecasting](https://mlsanity.com/paper/2605.10822/) (2026)

## Abstract (arXiv)

Understanding the robustness of deep learning models for multivariate long-term time series forecasting (M-LTSF) remains challenging, as evaluations typically rely on real-world datasets with unknown noise properties. We propose a simulation-based evaluation framework that generates parameterizable synthetic datasets, where each dataset instance corresponds to a different configuration of signal components, noise types, signal-to-noise ratios, and frequency characteristics. These configurable components aim to model real-world multivariate time series data without the ambiguity of unknown noise. This framework enables fine-grained, systematic evaluation of M-LTSF models under controlled and diverse scenarios. We benchmark four representative architectures S-Mamba (state-space), iTransformer (transformer-based), R-Linear (linear), and Autoformer (decomposition-based). Our analysis reveals that all models degrade severely when lookback windows cannot capture complete periods of seasonal patters in the data. S-Mamba and Autoformer perform best on sawtooth patterns, while R-Linear and iTransformer favor sinusoidal signals. White and Brownian noise universally degrade performance with lower signal-to-noise ratio while S-Mamba shows specific trend-noise and iTransformer shows seasonal-noise vulnerability. Further spectral analysis shows that S-Mamba and iTransformer achieve superior frequency reconstruction. This controlled approach, based on our synthetic and principle-driven testbed, offers deeper insights into model-specific strengths and limitations through the aggregation of MSE scores and provides concrete guidance for model selection based on signal characteristics and noise conditions.