mlsanityData to 7 Oct 2026

Alchemist: Towards the Design of Efficient Online Continual Learning System

arXiv 2503.01066 ↗First posted 3 Mar 2025cs.LG cs.CL cs.DCcited by 0

Not on the map. Why?

Abstract (arXiv)

Continual learning has become a promising solution to refine large language models incrementally by leveraging user feedback. In particular, online continual learning - iteratively training the model with small batches of user feedback - has demonstrated notable performance improvements. However, the existing practice of separating training and serving processes forces the online trainer to recompute the intermediate results already done during serving. Such redundant computations can account for 30%-42% of total training time. In this paper, we propose Alchemist, to the best of our knowledge, the first online continual learning system that efficiently reuses serving activations to increase training throughput. Alchemist introduces two key techniques: (1) recording and storing activations and KV cache only during the prefill phase to minimize latency and memory overhead; and (2) smart activation offloading and hedging. Evaluations with inputs of varied token length sampled from ShareGPT dataset show that compared with a separate training cluster, Alchemist significantly increases training throughput by up to 1.72x, reduces up to 47% memory usage during training, and supports up to 2x more training tokens - all while maintaining negligible impact on serving latency.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 54 of its 83 references that are in the map. First posted on arXiv 3 Mar 2025.

Builds on

54 of 83 references.

References · 54

  1. 2017Attention Is All You Needcited by 59,932
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  5. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  6. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  7. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  8. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
  9. 2020Learning to summarize from human feedbackcited by 2,562
  10. 2019Fine-Tuning Language Models from Human Preferencescited by 2,124
  11. 2020Don't Stop Pretraining: Adapt Language Models to Domains and Taskscited by 1,548
  12. 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,124
38 more
  1. 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,107
  2. 2021A General Language Assistant as a Laboratory for Alignmentcited by 1,023
  3. 2023BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Datasetcited by 828
  4. 2023RLAIF vs. RLHF: Scaling Reinforcement Learning from Human Feedback with AI Feedbackcited by 615
  5. 2024DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Servingcited by 595
  6. 2023Splitwise: Efficient generative LLM inference using phase splittingcited by 536
  7. 2020Artificial Intelligence, Values and Alignmentcited by 443
  8. 2023SpecInfer: Accelerating Generative Large Language Model Serving with Tree-based Speculative Inference and Verificationcited by 359
  9. 2023VeRA: Vector-based Random Matrix Adaptationcited by 294
  10. 2024Direct Language Model Alignment from Online AI Feedbackcited by 257
  11. 2023Atom: Low-bit Quantization for Efficient and Accurate LLM Servingcited by 255
  12. 2025FlashInfer: Efficient and Customizable Attention Engine for LLM Inference Servingcited by 248
  13. 2024InfiniGen: Efficient Generative Inference of Large Language Models with Dynamic KV Cache Managementcited by 242
  14. 2019ERNIE 2.0: A Continual Pre-training Framework for Language Understandingcited by 230
  15. 2024QServe: W4A8KV4 Quantization and System Co-design for Efficient LLM Servingcited by 200
  16. 2023CacheGen: KV Cache Compression and Streaming for Fast Large Language Model Servingcited by 199
  17. 2023Beyond Reverse KL: Generalizing Direct Preference Optimization with Diverse Divergence Constraintscited by 174
  18. 2021Alignment of Language Agentscited by 167
  19. 2023Aligning Language Models with Preferences through f-divergence Minimizationcited by 118
  20. 2025Towards System 2 Reasoning in LLMs: Learning How to Think With Meta Chain-of-Thoughtcited by 108
  21. 2024Parrot: Efficient Serving of LLM-based Applications with Semantic Variablecited by 96
  22. 2022CERT: Continual Pre-Training on Sketches for Library-Oriented Code Generationcited by 87
  23. 2022Continual Pre-Training Mitigates Forgetting in Language and Visioncited by 82
  24. 2024Preble: Efficient Distributed Prompt Scheduling for LLM Servingcited by 68
  25. 2024Aligning LLM Agents by Learning Latent Preference from User Editscited by 66
  26. 2020Ekya: Continuous Learning of Video Analytics Models on Edge Compute Serverscited by 58
  27. 2025Optimizing Large Language Model Training Using FP4 Quantizationcited by 53
  28. 2023Adapting Large Language Models to Domains via Reading Comprehensioncited by 48
  29. 2024Optimizing RLHF Training for Large Language Models with Stage Fusioncited by 41
  30. 2024Online DPO: Online Direct Preference Optimization with Fast-Slow Chasingcited by 38
  31. 2024Aligning Language Models with Demonstrated Feedbackcited by 32
  32. 2024Filtered Direct Preference Optimizationcited by 24
  33. 2024ByteCheckpoint: A Unified Checkpointing System for Large Foundation Model Developmentcited by 23
  34. 2024MicroScopiQ: Accelerating Foundational Models through Outlier-Aware Microscaling Quantizationcited by 22
  35. 2023ConPET: Continual Parameter-Efficient Tuning for Large Language Modelscited by 16
  36. 2025Locality-aware Fair Scheduling in LLM Servingcited by 15
  37. 2024Minder: Faulty Machine Detection for Large-scale Distributed Model Trainingcited by 11
  38. 2024FlexLLM: Token-Level Co-Serving of LLM Inference and Finetuning with SLO Guaranteescited by 9

50 most cited of 54 All 54 (JSON)

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Continual Learning for Large Language Models: A Survey
  2. 2024SwitchCIT: Switching for Continual Instruction Tuning
  3. 2023TRACE: A Comprehensive Benchmark for Continual Learning in Large Language Models
  4. 2026Learning without Overwriting: A Theory of Self-Distillation and Supervised Fine-Tuning in Continual Reasoning
  5. 2025SWE-Bench-CL: Continual Learning for Coding Agents
  6. 2024CoIN: A Benchmark of Continual Instruction tuNing for Multimodel Large Language Model
  7. 2025MCITlib: Multimodal Continual Instruction Tuning Library and Benchmark
  8. 2025Reinforced Interactive Continual Learning via Real-time Noisy Human Feedback