# BlackGoose Rimer: Harnessing RWKV-7 as a Simple yet Superior Replacement for Transformers in Large-Scale Time Series Modeling

arXiv: https://arxiv.org/abs/2503.06121 · HTML: https://mlsanity.com/paper/2503.06121/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 13 references that are in the map.
- First posted on arXiv 8 Mar 2025.

## Builds on (most cited first, 8 of 8)

- [Deep Equilibrium Models](https://mlsanity.com/paper/1909.01377/) (2019)
- [Learning to (Learn at Test Time): RNNs with Expressive Hidden States](https://mlsanity.com/paper/2407.04620/) (2024)
- [Timer: Generative Pre-trained Transformers Are Large Time Series Models](https://mlsanity.com/paper/2402.02368/) (2024)
- [Unlocking State-Tracking in Linear RNNs Through Negative Eigenvalues](https://mlsanity.com/paper/2411.12537/) (2024)
- [RWKV-TS: Beyond Traditional Recurrent Neural Network for Time Series Tasks](https://mlsanity.com/paper/2401.09093/) (2024)
- [ARWKV: Pretrain is not what we need, an RNN-Attention-Based Language Model Born from Transformer](https://mlsanity.com/paper/2501.15570/) (2025)
- [DFT: A Dual-branch Framework of Fluctuation and Trend for Stock Price Prediction](https://mlsanity.com/paper/2411.06065/) (2024)
- [TabulaTime: A Novel Multimodal Deep Learning Framework for Advancing Acute Coronary Syndrome Prediction through Environmental and Clinical Data Integration](https://mlsanity.com/paper/2502.17049/) (2025)

## Built on by (most cited first, 1 of 1)

- [RWKVTTS: Yet another TTS based on RWKV-7](https://mlsanity.com/paper/2504.03289/) (2025)

## Abstract (arXiv)

Time series models face significant challenges in scaling to handle large and complex datasets, akin to the scaling achieved by large language models (LLMs). The unique characteristics of time series data and the computational demands of model scaling necessitate innovative approaches. While researchers have explored various architectures such as Transformers, LSTMs, and GRUs to address these challenges, we propose a novel solution using RWKV-7, which incorporates meta-learning into its state update mechanism. By integrating RWKV-7's time mix and channel mix components into the transformer-based time series model Timer, we achieve a substantial performance improvement of approximately 1.13 to 43.3x and a 4.5x reduction in training time with 1/23 parameters, all while utilizing fewer parameters. Our code and model weights are publicly available for further research and development at https://github.com/Alic-Li/BlackGoose_Rimer.