mlsanityData to 14 Oct 2026

Scaling Up On-Device LLMs via Active-Weight Swapping Between DRAM and Flash

arXiv 2504.08378 ↗First posted 11 Apr 2025cs.LGcited by 4

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) are increasingly being deployed on mobile devices, but the limited DRAM capacity constrains the deployable model size. This paper introduces ActiveFlow, the first LLM inference framework that can achieve adaptive DRAM usage for modern LLMs (not ReLU-based), enabling the scaling up of deployable model sizes. The framework is based on the novel concept of active weight DRAM-flash swapping and incorporates three novel techniques: (1) Cross-layer active weights preloading. It uses the activations from the current layer to predict the active weights of several subsequent layers, enabling computation and data loading to overlap, as well as facilitating large I/O transfers. (2) Sparsity-aware self-distillation. It adjusts the active weights to align with the dense-model output distribution, compensating for approximations introduced by contextual sparsity. (3) Active weight DRAM-flash swapping pipeline. It orchestrates the DRAM space allocation among the hot weight cache, preloaded active weights, and computation-involved weights based on available memory. Results show ActiveFlow achieves the performance-cost Pareto frontier compared to existing efficiency optimization methods.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 22 of its 29 references that are in the map. First posted on arXiv 11 Apr 2025.

Builds on

22 of 29 references.

References · 22

  1. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  2. 2023SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shotcited by 1,121
  3. 2023FlexGen: High-Throughput Generative Inference of Large Language Models with a Single GPUcited by 575
  4. 2025Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attentioncited by 407
  5. 2023Deja Vu: Contextual Sparsity for Efficient LLMs at Inference Timecited by 252
  6. 2024InfiniGen: Efficient Generative Inference of Large Language Models with Dynamic KV Cache Managementcited by 245
  7. 2023PowerInfer: Fast Large Language Model Serving with a Consumer-grade GPUcited by 217
  8. 2023LLM in a flash: Efficient Large Language Model Inference with Limited Memorycited by 177
  9. 2024SeerAttention: Learning Intrinsic Sparse Attention in Your LLMscited by 124
  10. 2024PowerInfer-2: Fast Large Language Model Inference on a Smartphonecited by 103
  11. 2024BitDistiller: Unleashing the Potential of Sub-4-Bit LLMs via Self-Distillationcited by 84
  12. 2024T-MAC: CPU Renaissance via Table Lookup for Low-Bit LLM Deployment on Edgecited by 55
10 more
  1. 2024Training-Free Activation Sparsity in Large Language Modelscited by 55
  2. 2024Turbo Sparse: Achieving LLM SOTA Performance with Minimal Activated Parameterscited by 38
  3. 2024DB-LLM: Accurate Dual-Binarization for Efficient LLMscited by 37
  4. 2024Q-Sparse: All Large Language Models can be Fully Sparsely-Activatedcited by 25
  5. 2025Bitnet.cpp: Efficient Edge Inference for Ternary LLMscited by 22
  6. 2023Poster: Self-Supervised Quantization-Aware Knowledge Distillationcited by 11
  7. 2024CFSP: An Efficient Structured Pruning Framework for LLMs with Coarse-to-Fine Activation Informationcited by 11
  8. 2022Boosting DNN Cold Inference on Edge Devicescited by 6
  9. 2024Neuralink: Fast LLM Inference on Smartphones with Neuron Co-Activation Linkingcited by 5
  10. 2025Regularization-based Framework for Quantization-, Fault- and Variability-Aware Trainingcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 3.

4 papers.

  1. 2025DynaKV: Enabling Accurate and Efficient Long-Sequence LLM Decoding on Smartphonescited by 4
  2. 2026Chain-based Distillation for Effective Initialization of Variable-Sized Small Language Modelscited by 0
  3. 2026JustFit: Just-in-Time State Management for Local LLM Servingcited by 0
  4. 2026Activation Sparsity with Weight Approximation for Faster LLM Decoding on Offloaded Weightscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Efficient LLM Inference using Dynamic Input Pruning and Cache-Aware Masking
  2. 2025MVDRAM: Enabling GeMV Execution in Unmodified DRAM for Low-Bit LLM Acceleration
  3. 2026LeanStream: A Speculate-and-Refine Streaming Framework for Efficient on-Device LLM Inference
  4. 2026mzCache: On-Device LLM Memory Management under Multitasking
  5. 2025MNN-LLM: A Generic Inference Engine for Fast Large Language Model Deployment on Mobile Devices
  6. 2025Architecting Long-Context LLM Acceleration with Packing-Prefetch Scheduler and Ultra-Large Capacity On-Chip Memories
  7. 2026Lever: Speculative LLM Inference on Smartphones
  8. 2026CHIPSMORE: Compute-in-Interconnect and -Memory Chiplets for Multi-Mode Multi-Request LLM Inference Acceleration