mlsanityData to 7 Oct 2026

SPA: Towards A Computational Friendly Cloud-Base and On-Devices Collaboration Seq2seq Personalized Generation with Casual Inference

arXiv 2403.07088 ↗First posted 11 Mar 2024cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Large language models(LLMs) have shown its outperforming ability on various tasks and question answering. However, LLMs require substantial memory storage on low-resource devices. More critically, the computational speed on these devices is also severely limited. In this paper, we propose SPA(Side Plugin Adaption), a lightweight architecture for fast on-devices inference on the constraints of strict on-devices computation and memory constraints. Compared with other on-devices seq2seq generation, SPA could make a fast and stable inference on low-resource constraints, allowing it to obtain cost effiency. Our method establish an interaction between a pretrained LLMs on-cloud and additive parameters on-devices, which could provide the knowledge on both pretrained LLMs and featured personal feature. Further more, SPA provides a framework to keep feature-base parameters on low computational devices while leave the parameters containing general information on the high computational devices.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 34 of its 64 references that are in the map. First posted on arXiv 11 Mar 2024.

Builds on

34 of 64 references.

References · 34

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2023GPT-4 Technical Reportcited by 18,132
  4. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  5. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  6. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  7. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  8. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  9. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
  10. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,838
  11. 2018Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarizationcited by 1,308
  12. 2021GLM: General Language Model Pretraining with Autoregressive Blank Infillingcited by 1,026
22 more
  1. 2022Token Merging: Your ViT But Fastercited by 995
  2. 2023A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivitycited by 888
  3. 2021Towards a Unified View of Parameter-Efficient Transfer Learningcited by 863
  4. 2022GLM-130B: An Open Bilingual Pre-trained Modelcited by 837
  5. 2017Crowdsourcing Multiple Choice Science Questionscited by 742
  6. 2018CoQA: A Conversational Question Answering Challengecited by 674
  7. 2022Efficiently Scaling Transformer Inferencecited by 529
  8. 2023LLM-Adapters: An Adapter Family for Parameter-Efficient Fine-Tuning of Large Language Modelscited by 317
  9. 2020Knowledge Distillation and Student-Teacher Learning for Visual Intelligence: A Review and New Outlookscited by 312
  10. 2022LST: Ladder Side-Tuning for Parameter and Memory Efficient Transfer Learningcited by 228
  11. 2023LoftQ: LoRA-Fine-Tuning-Aware Quantization for Large Language Modelscited by 195
  12. 2023LLM in a flash: Efficient Large Language Model Inference with Limited Memorycited by 170
  13. 2020Exploring Versatile Generative Language Model Via Parameter-Efficient Transfer Learningcited by 139
  14. 2021Learned Token Pruning for Transformerscited by 138
  15. 2022Parameter-Efficient and Student-Friendly Knowledge Distillationcited by 47
  16. 2023LLMCad: Fast and Scalable On-device Large Language Model Inferencecited by 45
  17. 2021Subformer: Exploring Weight Sharing for Parameter Efficiency in Generative Transformerscited by 42
  18. 2022Enable Deep Learning on Mobile Devices: Methods, Systems, and Applicationscited by 32
  19. 2022EdgeFormer: A Parameter-Efficient Transformer for On-Device Seq2seq Generationcited by 21
  20. 2022Plug-and-Play Adaptation for Continuously-updated QAcited by 21
  21. 2023Cloud-Device Collaborative Learning for Multimodal Large Language Modelscited by 11
  22. 2021Greenformers: Improving Computation and Memory Efficiency in Transformer Models via Low-Rank Approximationcited by 5

Built on by

Not cited yet.