# Foundation Models Knowledge Distillation For Battery Capacity Degradation Forecast

arXiv: https://arxiv.org/abs/2505.08151 · HTML: https://mlsanity.com/paper/2505.08151/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in General knowledge distillation methods, one of 12 directions in Knowledge distillation.
- Builds on 24 of its 50 references that are in the map.
- First posted on arXiv 13 May 2025.

## Placed in

- Knowledge distillation / [General knowledge distillation methods](https://mlsanity.com/distill/general-knowledge-distillation-methods/)

## Builds on (most cited first, 24 of 24)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [A Time Series is Worth 64 Words: Long-term Forecasting with Transformers](https://mlsanity.com/paper/2211.14730/) (2022)
- [Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting](https://mlsanity.com/paper/2106.13008/) (2021)
- [Knowledge Distillation: A Survey](https://mlsanity.com/paper/2006.05525/) (2020)
- [Are Transformers Effective for Time Series Forecasting?](https://mlsanity.com/paper/2205.13504/) (2022)
- [FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecasting](https://mlsanity.com/paper/2201.12740/) (2022)
- [TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysis](https://mlsanity.com/paper/2210.02186/) (2022)
- [iTransformer: Inverted Transformers Are Effective for Time Series Forecasting](https://mlsanity.com/paper/2310.06625/) (2023)
- [One Fits All:Power General Time Series Analysis by Pretrained LM](https://mlsanity.com/paper/2302.11939/) (2023)
- [TimeMixer: Decomposable Multiscale Mixing for Time Series Forecasting](https://mlsanity.com/paper/2405.14616/) (2024)
- [Long-term Forecasting with TiDE: Time-series Dense Encoder](https://mlsanity.com/paper/2304.08424/) (2023)
- [TSMixer: An All-MLP Architecture for Time Series Forecasting](https://mlsanity.com/paper/2303.06053/) (2023)
- [Timer: Generative Pre-trained Transformers Are Large Time Series Models](https://mlsanity.com/paper/2402.02368/) (2024)
- [Frequency-domain MLPs are More Effective Learners in Time Series Forecasting](https://mlsanity.com/paper/2311.06184/) (2023)
- [Less Is More: Fast Multivariate Time Series Forecasting with Light Sampling-oriented MLP Structures](https://mlsanity.com/paper/2207.01186/) (2022)
- [TimeGPT-1](https://mlsanity.com/paper/2310.03589/) (2023)
- [FiLM: Frequency improved Legendre Memory Model for Long-term Time Series Forecasting](https://mlsanity.com/paper/2205.08897/) (2022)
- [Lag-Llama: Towards Foundation Models for Probabilistic Time Series Forecasting](https://mlsanity.com/paper/2310.08278/) (2023)
- [SegRNN: Segment Recurrent Neural Network for Long-Term Time Series Forecasting](https://mlsanity.com/paper/2308.11200/) (2023)
- [FilterNet: Harnessing Frequency Filters for Time Series Forecasting](https://mlsanity.com/paper/2411.01623/) (2024)
- [Beyond LoRA: Exploring Efficient Fine-Tuning Techniques for Time Series Foundational Models](https://mlsanity.com/paper/2409.11302/) (2024)
- [Automated feature extraction and selection for data-driven models of rapid battery capacity fade and end of life](https://mlsanity.com/paper/2101.04440/) (2021)

## Built on by (most cited first, 2 of 2)

- [Post-Training in Time Series Foundation Models: A Unifying Framework](https://mlsanity.com/paper/2607.20002/) (2026)
- [Knowledge Distillation for Efficient Transformer-Based Reinforcement Learning in Hardware-Constrained Energy Management Systems](https://mlsanity.com/paper/2603.26249/) (2026)

## Abstract (arXiv)

Accurate forecasting of lithium-ion battery capacity degradation is critical for reliable and safe operation, yet remains challenging under distribution shifts across scales and operating regimes. Here we investigate a time-series foundation model, that is, a large pre-trained time-series model for capacity degradation forecasting, and propose a degradation-aware fine-tuning strategy that aligns the model to capacity trajectories while retaining broadly transferable temporal structure. We instantiate this approach by fine-tuning the Timer model on 220,153 cycles of open-source charge-discharge records to obtain Battery-Timer. Using our released CycleLife-SJTUIE dataset, a real-world industrial collection from an energy-storage station with long-horizon cycling, we evaluate capacity generalization from small cells to large-scale storage systems and across varying operating conditions. Battery-Timer consistently outperforms specialized expert models. To address deployment cost, we further introduce knowledge distillation, a teacher-student transfer that compresses the foundation model's behavior into compact expert models. Distillation across several state-of-the-art time-series experts improves multi-condition capacity generalization while substantially reducing computational overhead, indicating a practical path to deployable cross-scale degradation forecasting by combining a foundation model with targeted distillation.