# Pretrained battery transformer (PBT): A foundation model for battery life prediction

arXiv: https://arxiv.org/abs/2512.16334 · HTML: https://mlsanity.com/paper/2512.16334/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 69 references that are in the map.
- First posted on arXiv 18 Dec 2025.

## Builds on (most cited first, 16 of 16)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)
- [A Survey on Mixture of Experts in Large Language Models](https://mlsanity.com/paper/2407.06204/) (2024)
- [A Survey on All-in-One Image Restoration: Taxonomy, Evaluation and Future Trends](https://mlsanity.com/paper/2410.15067/) (2024)
- [Learning from models beyond fine-tuning](https://mlsanity.com/paper/2310.08184/) (2023)
- [Learning Domain Invariant Prompt for Vision-Language Models](https://mlsanity.com/paper/2212.04196/) (2022)
- [Predicting the impact of formation protocols on battery lifetime immediately after manufacturing](https://mlsanity.com/paper/2203.14158/) (2022)
- [Predicting Battery Lifetime Under Varying Usage Conditions from Early Aging Data](https://mlsanity.com/paper/2307.08382/) (2023)
- [PulseBat: A field-accessible dataset for second-life battery diagnostics from realistic histories using multidimensional rapid pulse test](https://mlsanity.com/paper/2502.16848/) (2025)
- [Forecasting battery capacity and power degradation with multi-task learning](https://mlsanity.com/paper/2111.14937/) (2021)
- [Systematic Feature Design for Cycle Life Prediction of Lithium-Ion Batteries During Formation](https://mlsanity.com/paper/2410.07458/) (2024)
- [Physics-informed mixture of experts network for interpretable battery degradation trajectory computation amid second-life complexities](https://mlsanity.com/paper/2506.17755/) (2025)

## Abstract (arXiv)

Early prediction of battery cycle life is essential for improving battery design, manufacturing and deployment. However, despite encouraging progress with machine learning, battery life prediction remains constrained by scarce data and pronounced heterogeneity across battery chemistries, specifications, formation protocols and operating conditions. Although transfer learning has been widely explored to alleviate these challenges, its effectiveness is limited by the absence of a foundation model that can integrate heterogeneous battery life data and provide broadly useful knowledge for target-scenario specialization. Here we introduce the pretrained battery transformer (PBT), an integrated foundation model comprising a general PBT and specialized PBT models for individual target scenarios. At its core, battery-knowledge-encoded mixture-of-experts layers enable the general PBT to consolidate shared cycling-pattern-lifetime relationships from 13 heterogeneous lithium-ion battery datasets while preserving specialization across distinct aging regimes. The resulting general PBT provides a shared knowledge and parameter foundation from which specialized PBT models are constructed using limited labelled data to capture target-specific degradation behavior. Across 15 downstream datasets covering 977 batteries and 532 aging conditions from lithium-ion, sodium-ion and zinc-ion batteries, the specialized PBT models achieve state-of-the-art performance, outperforming the strongest comparator by 24.8% on average and by up to 73.9%. This study establishes, to our knowledge, the first foundation model for battery life prediction and points towards a shift from isolated, scenario-specific modelling to a reusable knowledge foundation for data-efficient specialization, with broader implications for sustainable-energy prediction problems constrained by scarce and heterogeneous data.