mlsanityData to 7 Oct 2026

Foundation Models Knowledge Distillation For Battery Capacity Degradation Forecast

arXiv 2505.08151 ↗First posted 13 May 2025cs.AIcited by 2

Not on the map. Why?

Abstract (arXiv)

Accurate forecasting of lithium-ion battery capacity degradation is critical for reliable and safe operation, yet remains challenging under distribution shifts across scales and operating regimes. Here we investigate a time-series foundation model, that is, a large pre-trained time-series model for capacity degradation forecasting, and propose a degradation-aware fine-tuning strategy that aligns the model to capacity trajectories while retaining broadly transferable temporal structure. We instantiate this approach by fine-tuning the Timer model on 220,153 cycles of open-source charge-discharge records to obtain Battery-Timer. Using our released CycleLife-SJTUIE dataset, a real-world industrial collection from an energy-storage station with long-horizon cycling, we evaluate capacity generalization from small cells to large-scale storage systems and across varying operating conditions. Battery-Timer consistently outperforms specialized expert models. To address deployment cost, we further introduce knowledge distillation, a teacher-student transfer that compresses the foundation model's behavior into compact expert models. Distillation across several state-of-the-art time-series experts improves multi-condition capacity generalization while substantially reducing computational overhead, indicating a practical path to deployable cross-scale degradation forecasting by combining a foundation model with targeted distillation.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 24 of its 50 references that are in the map. First posted on arXiv 13 May 2025.

Builds on

24 of 50 references.

References · 24

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  3. 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,120
  4. 2022A Time Series is Worth 64 Words: Long-term Forecasting with Transformerscited by 1,981
  5. 2021Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecastingcited by 1,654
  6. 2020Knowledge Distillation: A Surveycited by 1,578
  7. 2022Are Transformers Effective for Time Series Forecasting?cited by 1,400
  8. 2022FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecastingcited by 1,200
  9. 2022TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysiscited by 1,030
  10. 2023iTransformer: Inverted Transformers Are Effective for Time Series Forecastingcited by 906
  11. 2023One Fits All:Power General Time Series Analysis by Pretrained LMcited by 615
  12. 2024TimeMixer: Decomposable Multiscale Mixing for Time Series Forecastingcited by 333
12 more
  1. 2023Long-term Forecasting with TiDE: Time-series Dense Encodercited by 327
  2. 2023TSMixer: An All-MLP Architecture for Time Series Forecastingcited by 232
  3. 2024Timer: Generative Pre-trained Transformers Are Large Time Series Modelscited by 198
  4. 2023Frequency-domain MLPs are More Effective Learners in Time Series Forecastingcited by 170
  5. 2022Less Is More: Fast Multivariate Time Series Forecasting with Light Sampling-oriented MLP Structurescited by 158
  6. 2023TimeGPT-1cited by 149
  7. 2022FiLM: Frequency improved Legendre Memory Model for Long-term Time Series Forecastingcited by 147
  8. 2023Lag-Llama: Towards Foundation Models for Probabilistic Time Series Forecastingcited by 102
  9. 2023SegRNN: Segment Recurrent Neural Network for Long-Term Time Series Forecastingcited by 53
  10. 2024FilterNet: Harnessing Frequency Filters for Time Series Forecastingcited by 44
  11. 2024Beyond LoRA: Exploring Efficient Fine-Tuning Techniques for Time Series Foundational Modelscited by 10
  12. 2021Automated feature extraction and selection for data-driven models of rapid battery capacity fade and end of lifecited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.