mlsanityData to 7 Oct 2026

Backbone Augmented Training for Adaptations

arXiv 2506.04288 ↗First posted 4 Jun 2025cs.LGcited by 0

On the map: Diffusion models → Image generation, text-to-image and editing

Abstract (arXiv)

Adaptations facilitate efficient training of large backbone models, including diffusion models for image generation and transformer-based language models. While various adaptation techniques enhance performance with minimal computational resources, limited adaptation data often leads to challenges in training. To address this, we focus on the enormous amount of backbone data used to pre-train the backbone models. We propose Backbone Augmented Training (BAT), a method that leverages backbone data to augment the adaptation dataset. First, we formulate and prove two mathematical key propositions: one establishes the validity of BAT, while the other identifies a condition under which BAT benefits adaptation. Furthermore, we introduce an advanced data selection scheme that satisfies these propositions and present ALBAT algorithm to implement this approach. ALBAT efficiently enhances adaptation training in both personalization and language generation tasks with scarce data.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 30 of its 42 references that are in the map. First posted on arXiv 4 Jun 2025.

Builds on

30 of 42 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023Navigating Text-To-Image Customization: From LyCORIS Fine-Tuning to Model Evaluation26 Sep

Other references · 29

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  5. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  6. 2022Scalable Diffusion Models with Transformerscited by 5,931
  7. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  8. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
  9. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  10. 2020Don't Stop Pretraining: Adapt Language Models to Domains and Taskscited by 1,548
  11. 2018Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarizationcited by 1,308
  12. 2021BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-modelscited by 1,234
17 more
  1. 2024DoRA: Weight-Decomposed Low-Rank Adaptationcited by 740
  2. 2020Intrinsic Dimensionality Explains the Effectiveness of Language Model Fine-Tuningcited by 658
  3. 2021Compacter: Efficient Low-Rank Hypercomplex Adapter Layerscited by 458
  4. 2014New insights and perspectives on the natural gradient methodcited by 435
  5. 2020Parameter-Efficient Transfer Learning with Diff Pruningcited by 351
  6. 2019A Question-Entailment Approach to Question Answeringcited by 133
  7. 2023InfoBatch: Lossless Training Speed Up by Unbiased Dynamic Data Pruningcited by 90
  8. 2013A Statistical Perspective on Algorithmic Leveragingcited by 38
  9. 2021Improved Fine-Tuning by Better Leveraging Pre-Training Datacited by 36
  10. 2019Overfitting Mechanism and Avoidance in Deep Neural Networkscited by 30
  11. 2019Less Is Better: Unweighted Data Subsampling via Influence Functioncited by 30
  12. 2024Rethinking Data Shapley for Data Selection Tasks: Misleads and Meritscited by 29
  13. 2023Towards a statistical theory of data selection under weak supervisioncited by 25
  14. 2017Optimal Sub-sampling with Influence Functionscited by 11
  15. 2018Optimal Subsampling Algorithms for Big Data Regressionscited by 10
  16. 2023Does Synthetic Data Make Large Language Models More Efficient?cited by 8
  17. 2025Aligning Instruction Tuning with Pre-trainingcited by 7

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024CAT: Contrastive Adapter Training for Personalized Image Generation
  2. 2024Scalable Multi-Domain Adaptation of Language Models using Modular Experts
  3. 2026To Adapt or Not to Adapt? Selective Adaptation for Vision-Language Models
  4. 2024On Domain-Adaptive Post-Training for Multimodal Large Language Models
  5. 2025From Next-Token to Next-Block: A Principled Adaptation Path for Diffusion LLMs
  6. 2024Understanding Retrieval-Augmented Task Adaptation for Vision-Language Models
  7. 2023Orthogonal Adaptation for Modular Customization of Diffusion Models
  8. 2026RAISE: Requirement-Adaptive Evolutionary Refinement for Training-Free Text-to-Image Alignment