mlsanityData to 14 Oct 2026

Quantum-PEFT: Ultra parameter-efficient fine-tuning

arXiv 2503.05431 ↗First posted 7 Mar 2025cs.LGcited by 11

Not on the map. Why?

Abstract (arXiv)

This paper introduces Quantum-PEFT that leverages quantum computations for parameter-efficient fine-tuning (PEFT). Unlike other additive PEFT methods, such as low-rank adaptation (LoRA), Quantum-PEFT exploits an underlying full-rank yet surprisingly parameter efficient quantum unitary parameterization. With the use of Pauli parameterization, the number of trainable parameters grows only logarithmically with the ambient dimension, as opposed to linearly as in LoRA-based PEFT methods. Quantum-PEFT achieves vanishingly smaller number of trainable parameters than the lowest-rank LoRA as dimensions grow, enhancing parameter efficiency while maintaining a competitive performance. We apply Quantum-PEFT to several transfer learning benchmarks in language and vision, demonstrating significant advantages in parameter efficiency.

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Builds on 55 of its 82 references that are in the map. First posted on arXiv 7 Mar 2025.

Builds on

55 of 82 references.

References · 55

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2024The Llama 3 Herd of Modelscited by 14,715
  5. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  6. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,899
  7. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
  8. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,915
  9. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,795
  10. 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,181
  11. 2024Mixtral of Expertscited by 1,823
  12. 2021DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharingcited by 1,254
38 more
  1. 2021BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-modelscited by 1,245
  2. 2021Towards a Unified View of Parameter-Efficient Transfer Learningcited by 872
  3. 2020AdapterFusion: Non-Destructive Task Composition for Transfer Learningcited by 788
  4. 2022AdaptFormer: Adapting Vision Transformers for Scalable Visual Recognitioncited by 719
  5. 2020Intrinsic Dimensionality Explains the Effectiveness of Language Model Fine-Tuningcited by 671
  6. 2021Compacter: Efficient Low-Rank Hypercomplex Adapter Layerscited by 460
  7. 2024LoRA+: Efficient Low Rank Adaptation of Large Modelscited by 377
  8. 2015Unitary Evolution Recurrent Neural Networkscited by 294
  9. 2017The E2E Dataset: New Challenges For End-to-End Generationcited by 253
  10. 2015Tensorizing Neural Networkscited by 250
  11. 2023Controlling Text-to-Image Diffusion by Orthogonal Finetuningcited by 208
  12. 2024ReFT: Representation Finetuning for Language Modelscited by 197
  13. 2021FedPara: Low-Rank Hadamard Product for Communication-Efficient Federated Learningcited by 141
  14. 2022FacT: Factor-Tuning for Lightweight Adaptation on Vision Transformercited by 133
  15. 2022KronA: Parameter Efficient Tuning with Kronecker Adaptercited by 133
  16. 2018Quantum generative adversarial networkscited by 128
  17. 2023Parameter-Efficient Orthogonal Finetuning via Butterfly Factorizationcited by 114
  18. 2023Navigating Text-To-Image Customization: From LyCORIS Fine-Tuning to Model Evaluationcited by 109
  19. 2016Full-Capacity Unitary Recurrent Neural Networkscited by 94
  20. 2024LISA: Layerwise Importance Sampling for Memory-Efficient Large Language Model Fine-Tuningcited by 89
  21. 2024Asymmetry in Low-Rank Adapters of Foundation Modelscited by 86
  22. 2023One-for-All: Generalized LoRA for Parameter-Efficient Fine-tuningcited by 85
  23. 2017Orthogonal Recurrent Neural Networks with Scaled Cayley Transformcited by 83
  24. 2020Efficient Riemannian Optimization on the Stiefel Manifold via the Cayley Transformcited by 80
  25. 2018Towards Quantum Machine Learning with Tensor Networkscited by 76
  26. 2023Delta-LoRA: Fine-Tuning High-Rank Parameters with the Delta of Low-Rank Matricescited by 74
  27. 2016Tunable Efficient Unitary Neural Networks (EUNN) and their application to RNNscited by 65
  28. 2018Can We Gain More from Orthogonality Regularizations in Training Deep CNNs?cited by 62
  29. 2016Efficient Orthogonal Parametrisation of Recurrent Neural Networks Using Householder Reflectionscited by 57
  30. 2024Riemannian Preconditioned LoRA for Fine-Tuning Foundation Modelscited by 52
  31. 2022Low-rank lottery tickets: finding efficient low-rank neural networks via matrix differential equationscited by 43
  32. 2023Bridging Discrete and Backpropagation: Straight-Through and Beyondcited by 31
  33. 2024MoRA: High-Rank Updating for Parameter-Efficient Fine-Tuningcited by 31
  34. 2024LoTR: Low Tensor Rank Weight Adaptationcited by 29
  35. 2024QuanTA: Efficient High-Rank Fine-Tuning of LLMs with Quantum-Informed Tensor Adaptationcited by 23
  36. 2021Quantum Graph Convolutional Neural Networkscited by 18
  37. 2023Consolidator: Mergeable Adapter with Grouped Connections for Visual Adaptationcited by 18
  38. 2023Geometry-aware training of factorized layers in tensor Tucker formatcited by 12

50 most cited of 55 All 55 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 5; 2026: 6.

11 papers.

  1. 2025RefLoRA: Refactored Low-Rank Adaptation for Efficient Fine-Tuning of Large Modelscited by 9
  2. 2025Quantum-Enhanced LLM Efficient Fine Tuningcited by 6
  3. 2025MetaTT: A Global Tensor-Train Adapter for Parameter-Efficient Fine-Tuningcited by 4
  4. 2025μ-MoE: Test-Time Pruning as Micro-Grained Mixture-of-Expertscited by 3
  5. 2025You Only Measure Once: On Designing Single-Shot Quantum Machine Learning Modelscited by 2
  6. 2026EinSort: Sorting is All We Need for Tensorizing LLMcited by 0
  7. 2026When AI meets quantum information: A comprehensive reviewcited by 0
  8. 2026MQAdapter: Multi-Modal Quantum Adapter for Coarse-to-Fine VLM Fine-tuningcited by 0
  9. 2026Circuit Hypernetworks for Quantum-Augmented Diffusion Language Modelscited by 0
  10. 2026QuPID: Quantum Parameter-Efficient Input-Dependent Retrieval Adaptation for Medical RAGcited by 0
  11. 2026First-Order Steering: Translating Weight Adaptation into Activation Steeringcited by 0