mlsanityData to 7 Oct 2026

Recovering the Pre-Fine-Tuning Weights of Generative Models

arXiv 2402.10208 ↗First posted 15 Feb 2024cs.LG cs.CL cs.CRcited by 15

Not on the map. Why?

Abstract (arXiv)

The dominant paradigm in generative modeling consists of two steps: i) pre-training on a large-scale but unsafe dataset, ii) aligning the pre-trained model with human values via fine-tuning. This practice is considered safe, as no current method can recover the unsafe, pre-fine-tuning model weights. In this paper, we demonstrate that this assumption is often false. Concretely, we present Spectral DeTuning, a method that can recover the weights of the pre-fine-tuning model using a few low-rank (LoRA) fine-tuned models. In contrast to previous attacks that attempt to recover pre-fine-tuning capabilities, our method aims to recover the exact pre-fine-tuning weights. Our approach exploits this new vulnerability against large-scale models such as a personalized Stable Diffusion and an aligned Mistral.

Key facts. Cited by 15 papers in the map (data through 7 Oct 2026). Builds on 56 of its 65 references that are in the map. First posted on arXiv 15 Feb 2024.

Builds on

56 of 65 references.

References · 56

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  5. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  6. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  7. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  8. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  9. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  10. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  11. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  12. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
38 more
  1. 2017Deep reinforcement learning from human preferencescited by 3,846
  2. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  3. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
  4. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,286
  5. 2023Mistral 7Bcited by 3,065
  6. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  7. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,221
  8. 2023Code Llama: Open Foundation Models for Codecited by 2,105
  9. 2022Visual Prompt Tuningcited by 1,674
  10. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,565
  11. 2015Microsoft COCO Captions: Data Collection and Evaluation Servercited by 1,447
  12. 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillationcited by 1,207
  13. 2023mPLUG-Owl: Modularization Empowers Large Language Models with Multimodalitycited by 1,038
  14. 2019Machine Unlearningcited by 1,020
  15. 2022Red Teaming Language Models with Language Modelscited by 983
  16. 2022Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learningcited by 886
  17. 2021Towards a Unified View of Parameter-Efficient Transfer Learningcited by 863
  18. 2023Enhancing Chat Language Models by Scaling High-quality Instructional Conversationscited by 825
  19. 2023AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedbackcited by 738
  20. 2021GPT Understands, Toocited by 715
  21. 2022Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learnedcited by 702
  22. 2016Stealing Machine Learning Models via Prediction APIscited by 602
  23. 2021LiT: Zero-Shot Transfer with Locked-image text Tuningcited by 572
  24. 2023Zephyr: Direct Distillation of LM Alignmentcited by 498
  25. 2023Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervisioncited by 445
  26. 2019A Large-scale Study of Representation Learning with the Visual Task Adaptation Benchmarkcited by 419
  27. 2021Differentially Private Fine-tuning of Language Modelscited by 392
  28. 2023LoraHub: Efficient Cross-Task Generalization via Dynamic LoRA Compositioncited by 359
  29. 2023Are aligned neural networks adversarially aligned?cited by 305
  30. 2023Mix-of-Show: Decentralized Low-Rank Adaptation for Multi-Concept Customization of Diffusion Modelscited by 261
  31. 2023Break-A-Scene: Extracting Multiple Concepts from a Single Imagecited by 230
  32. 2023HyperDreamBooth: HyperNetworks for Fast Personalization of Text-to-Image Modelscited by 224
  33. 2015Coordinate Descent Algorithmscited by 218
  34. 2022SpaText: Spatio-Textual Representation for Controllable Image Generationcited by 217
  35. 2023ZipLoRA: Any Subject in Any Style by Effectively Merging LoRAscited by 194
  36. 2023Fundamental Limitations of Alignment in Large Language Modelscited by 152
  37. 2021FedPara: Low-Rank Hadamard Product for Communication-Efficient Federated Learningcited by 140
  38. 2023Multitask Prompt Tuning Enables Parameter-Efficient Transfer Learningcited by 115

50 most cited of 56 All 56 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 7; 2025: 6; 2026: 2.

15 papers.

Sort:

By area:

  1. 2024Stable Flow: Vital Layers for Training-Free Image Editingcited by 111
  2. 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deploymentcited by 110
  3. 2025Safety at Scale: A Comprehensive Survey of Large Model and Agent Safetycited by 63
  4. 2025We Should Chart an Atlas of All the World's Modelscited by 25
  5. 2024DiffUHaul: A Training-Free Method for Object Dragging in Imagescited by 22
  6. 2024Deep Linear Probe Generators for Weight Space Learningcited by 17
  7. 2024Unsupervised Model Tree Heritage Recoverycited by 11
  8. 2024Dataset Size Recovery from LoRA Weightscited by 7
  9. 2024Learning on Model Weights using Tree Expertscited by 7
  10. 2025Activation Approximations Can Incur Safety Vulnerabilities Even in Aligned LLMs: Comprehensive Analysis and Defensecited by 7
  11. 2024Unforgettable Generalization in Language Modelscited by 4
  12. 2025Does Low Rank Adaptation Lead to Lower Robustness against Training-Time Attacks?cited by 4
3 more
  1. 2025Position: It's Time to Act on the Risk of Efficient Personalized Text Generationcited by 1
  2. 2026Discovering Hidden Gems in Model Repositoriescited by 0
  3. 2026LoREnc: Low-Rank Encryption for Securing Foundation Models and LoRA Adapterscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Recursive Learning Without Collapse: A Weighting-Based Stabilization Framework
  2. 2024Risks When Sharing LoRA Fine-Tuned Diffusion Model Weights
  3. 2023On the Stability of Iterative Retraining of Generative Models on their own Data
  4. 2026Source-Prior-Driven Selective Adaptation for Efficient Diffusion Model Finetuning
  5. 2024Singular Value Scaling: Efficient Generative Model Compression via Pruned Weights Refinement
  6. 2024Efficient Fine-Tuning and Concept Suppression for Pruned Diffusion Models
  7. 2025Towards Irreversible Machine Unlearning for Diffusion Models
  8. 2024Spectrum-Aware Parameter Efficient Fine-Tuning for Diffusion Models