Foundation Models Knowledge Distillation For Battery Capacity Degradation Forecast
Not on the map. Why?
Abstract (arXiv)
Accurate forecasting of lithium-ion battery capacity degradation is critical for reliable and safe operation, yet remains challenging under distribution shifts across scales and operating regimes. Here we investigate a time-series foundation model, that is, a large pre-trained time-series model for capacity degradation forecasting, and propose a degradation-aware fine-tuning strategy that aligns the model to capacity trajectories while retaining broadly transferable temporal structure. We instantiate this approach by fine-tuning the Timer model on 220,153 cycles of open-source charge-discharge records to obtain Battery-Timer. Using our released CycleLife-SJTUIE dataset, a real-world industrial collection from an energy-storage station with long-horizon cycling, we evaluate capacity generalization from small cells to large-scale storage systems and across varying operating conditions. Battery-Timer consistently outperforms specialized expert models. To address deployment cost, we further introduce knowledge distillation, a teacher-student transfer that compresses the foundation model's behavior into compact expert models. Distillation across several state-of-the-art time-series experts improves multi-condition capacity generalization while substantially reducing computational overhead, indicating a practical path to deployable cross-scale degradation forecasting by combining a foundation model with targeted distillation.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 24 of its 50 references that are in the map. First posted on arXiv 13 May 2025.
Builds on
24 of 50 references.
References · 24
- 2017Attention Is All You Needcited by 59,932
- 2015Distilling the Knowledge in a Neural Networkcited by 10,076
- 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,120
- 2022A Time Series is Worth 64 Words: Long-term Forecasting with Transformerscited by 1,981
- 2021Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecastingcited by 1,654
- 2020Knowledge Distillation: A Surveycited by 1,578
- 2022Are Transformers Effective for Time Series Forecasting?cited by 1,400
- 2022FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecastingcited by 1,200
- 2022TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysiscited by 1,030
- 2023iTransformer: Inverted Transformers Are Effective for Time Series Forecastingcited by 906
- 2023One Fits All:Power General Time Series Analysis by Pretrained LMcited by 615
- 2024TimeMixer: Decomposable Multiscale Mixing for Time Series Forecastingcited by 333
12 more
- 2023Long-term Forecasting with TiDE: Time-series Dense Encodercited by 327
- 2023TSMixer: An All-MLP Architecture for Time Series Forecastingcited by 232
- 2024Timer: Generative Pre-trained Transformers Are Large Time Series Modelscited by 198
- 2023Frequency-domain MLPs are More Effective Learners in Time Series Forecastingcited by 170
- 2022Less Is More: Fast Multivariate Time Series Forecasting with Light Sampling-oriented MLP Structurescited by 158
- 2023TimeGPT-1cited by 149
- 2022FiLM: Frequency improved Legendre Memory Model for Long-term Time Series Forecastingcited by 147
- 2023Lag-Llama: Towards Foundation Models for Probabilistic Time Series Forecastingcited by 102
- 2023SegRNN: Segment Recurrent Neural Network for Long-Term Time Series Forecastingcited by 53
- 2024FilterNet: Harnessing Frequency Filters for Time Series Forecastingcited by 44
- 2024Beyond LoRA: Exploring Efficient Fine-Tuning Techniques for Time Series Foundational Modelscited by 10
- 2021Automated feature extraction and selection for data-driven models of rapid battery capacity fade and end of lifecited by 9
Built on by
Citing papers per year: 2026: 2.
2 papers.