# Parameter-Efficient Fine-Tuning for HAR: Integrating LoRA and QLoRA into Transformer Models

arXiv: https://arxiv.org/abs/2512.17983 · HTML: https://mlsanity.com/paper/2512.17983/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 19 of its 32 references that are in the map.
- First posted on arXiv 19 Dec 2025.

## Builds on (most cited first, 19 of 19)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Parameter-Efficient Transfer Learning for NLP](https://mlsanity.com/paper/1902.00751/) (2019)
- [QLoRA: Efficient Finetuning of Quantized LLMs](https://mlsanity.com/paper/2305.14314/) (2023)
- [Fine-Tuning Pretrained Language Models: Weight Initializations, Data Orders, and Early Stopping](https://mlsanity.com/paper/2002.06305/) (2020)
- [LoftQ: LoRA-Fine-Tuning-Aware Quantization for Large Language Models](https://mlsanity.com/paper/2310.08659/) (2023)
- [LoRA vs Full Fine-tuning: An Illusion of Equivalence](https://mlsanity.com/paper/2410.21228/) (2024)
- [A Survey on LoRA of Large Language Models](https://mlsanity.com/paper/2407.11046/) (2024)
- [The Ultimate Guide to Fine-Tuning LLMs from Basics to Breakthroughs: An Exhaustive Review of Technologies, Research, Best Practices, Applied Research Challenges and Opportunities](https://mlsanity.com/paper/2408.13296/) (2024)
- [Protecting Sensory Data against Sensitive Inferences](https://mlsanity.com/paper/1802.07802/) (2018)
- [LoRA-Whisper: Parameter-Efficient and Extensible Multilingual ASR](https://mlsanity.com/paper/2406.06619/) (2024)
- [Sparsely Shared LoRA on Whisper for Child Speech Recognition](https://mlsanity.com/paper/2309.11756/) (2023)
- [Low-Rank Adaptation of Time Series Foundational Models for Out-of-Domain Modality Forecasting](https://mlsanity.com/paper/2405.10216/) (2024)
- [Transformer-based Models to Deal with Heterogeneous Environments in Human Activity Recognition](https://mlsanity.com/paper/2209.11750/) (2022)
- [Combining Public Human Activity Recognition Datasets to Mitigate Labeled Data Scarcity](https://mlsanity.com/paper/2306.13735/) (2023)
- [ST-LoRA: Low-rank Adaptation for Spatio-Temporal Forecasting](https://mlsanity.com/paper/2404.07919/) (2024)
- [Evaluation and comparison of federated learning algorithms for Human Activity Recognition on smartphones](https://mlsanity.com/paper/2210.16918/) (2022)
- [Comparing Self-Supervised Learning Techniques for Wearable Human Activity Recognition](https://mlsanity.com/paper/2404.15331/) (2024)
- [AC-LoRA: Auto Component LoRA for Personalized Artistic Style Image Generation](https://mlsanity.com/paper/2504.02231/) (2025)

## Abstract (arXiv)

Human Activity Recognition is a foundational task in pervasive computing. While recent advances in self-supervised learning and transformer-based architectures have significantly improved HAR performance, adapting large pretrained models to new domains remains a practical challenge due to limited computational resources on target devices. This papers investigates parameter-efficient fine-tuning techniques, specifically Low-Rank Adaptation (LoRA) and Quantized LoRA, as scalable alternatives to full model fine-tuning for HAR. We propose an adaptation framework built upon a Masked Autoencoder backbone and evaluate its performance under a Leave-One-Dataset-Out validation protocol across five open HAR datasets. Our experiments demonstrate that both LoRA and QLoRA can match the recognition performance of full fine-tuning while significantly reducing the number of trainable parameters, memory usage, and training time. Further analyses reveal that LoRA maintains robust performance even under limited supervision and that the adapter rank provides a controllable trade-off between accuracy and efficiency. QLoRA extends these benefits by reducing the memory footprint of frozen weights through quantization, with minimal impact on classification quality.