mlsanityData to 14 Oct 2026

FBLayout: Optimizing Memory Layout for Efficient LLM Finetuning on Mobile GPUs

arXiv 2607.21624 ↗First posted 7 Jul 2026cs.AI cs.DC cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Transformer-based models have enabled unprecedented capabilities across language, vision, and multimodal tasks. On-device fine-tuning of transformer models offers a privacy-preserving path to personalized AI, yet remains inefficient on mobile GPUs due to severe memory constraints and frequent layout transformations in attention mechanism during training. Existing mobile training frameworks either use unified layouts for forward and backward passes -- leading to fragmented memory access and poor GPU utilization during backpropagation -- or rely on explicit layout conversions, which introduce significant transformation overhead. To overcome this, we propose FBLayout, a layout-aware framework that co-designs tensor organization with mobile GPU platforms. FBLayout introduces: (1) a unified R-Tile layout for multi-dimensional reductions across forward/backward passes; (2) tile-based index transformation to eliminate physical data movement; and (3) activation-guided layout selection to propagate efficient layouts globally. Evaluations on seven transformer models across different mobile phones (including ARM Mali and Qualcomm Adreno GPUs) show that FBLayout achieves 2.2-5.7x speedup over MNN, TFLite, and TVM, while significantly improving cache efficiency and reducing memory footprint, enabling practical on-device large model fine-tuning.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 38 of its 69 references that are in the map. First posted on arXiv 7 Jul 2026.

Builds on

38 of 69 references.

References · 38

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  5. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  6. 2024The Llama 3 Herd of Modelscited by 14,715
  7. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  8. 2025Qwen3 Technical Reportcited by 8,820
  9. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,795
  10. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
  11. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
  12. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,915
26 more
  1. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,568
  2. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
  3. 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,346
  4. 2021CrossViT: Cross-Attention Multi-Scale Vision Transformer for Image Classificationcited by 806
  5. 2022Vision Transformer Adapter for Dense Predictionscited by 565
  6. 20218-bit Optimizers via Block-wise Quantizationcited by 403
  7. 2023How Can Recommender Systems Benefit from Large Language Models: A Surveycited by 142
  8. 2024HydraLoRA: An Asymmetric LoRA Architecture for Efficient Fine-Tuningcited by 123
  9. 2024On-Device Language Models: A Comprehensive Reviewcited by 104
  10. 2024A Survey of Resource-efficient LLM and Multimodal Foundation Modelscited by 86
  11. 2020MNN: A Universal and Efficient Inference Enginecited by 65
  12. 2021DNNFusion: Accelerating Deep Neural Networks Execution with Advanced Operator Fusioncited by 57
  13. 2022Mandheling: Mixed-Precision On-Device DNN Training with DSP Offloadingcited by 29
  14. 2024Leveraging LLM Reasoning Enhances Personalized Recommender Systemscited by 28
  15. 2025CLONE: Customizing LLMs for Efficient Latency-Aware Inference at the Edgecited by 16
  16. 2023ElasticTrainer: Speeding Up On-Device Training with Runtime Elastic Tensor Selectioncited by 12
  17. 2024SmartMem: Layout Transformation Elimination and Adaptation for Efficient DNN Execution on Mobilecited by 12
  18. 2024Breaking the Memory Wall for Heterogeneous Federated Learning via Progressive Trainingcited by 10
  19. 2025MobiLLM: Enabling LLM Fine-Tuning on the Mobile Device via Server Assisted Side Tuningcited by 10
  20. 2025Elastic Mixture of Rank-Wise Experts for Knowledge Reuse in Federated Fine-Tuningcited by 6
  21. 2025Linear Layouts: Robust Code Generation of Efficient Tensor Computation Using mathbbF2cited by 4
  22. 2024Bridging Memory Gaps: Scaling Federated Learning for Heterogeneous Clientscited by 3
  23. 2025DAF: An Efficient End-to-End Dynamic Activation Framework for on-Device DNN Trainingcited by 3
  24. 2026Beyond End-to-End: Dynamic Chain Optimization for Private LLM Adaptation on the Edgecited by 3
  25. 2026Floe: Federated Specialization for Real-Time LLM-SLM Inferencecited by 2
  26. 2025FedHybrid: Breaking the Memory Wall of Federated Learning via Hybrid Tensor Managementcited by 1

Built on by

Not cited yet.