# Memory Analysis on the Training Course of DeepSeek Models

arXiv: https://arxiv.org/abs/2502.07846 · HTML: https://mlsanity.com/paper/2502.07846/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Feb 2025.

## Built on by (most cited first, 1 of 1)

- [TokenSeek: Memory Efficient Fine Tuning via Instance-Aware Token Ditching](https://mlsanity.com/paper/2601.19739/) (2026)

## Abstract (arXiv)

We present a theoretical analysis of GPU memory consumption during the training of DeepSeek models such as DeepSeek-v2 and DeepSeek-v3. Our primary objective is to clarify the device-level memory requirements associated with various distributed training configurations. Specifically, we examine critical factors influencing memory usage, including micro-batch size, activation recomputation policies, 3D parallelism, and ZeRO optimizations. It is important to emphasize that the training policies discussed in this report are not representative of DeepSeek's official configurations. Instead, they are explored to provide a deeper understanding of memory dynamics in training of large-scale mixture-of-experts model.