# Structural Priors and Modular Adapters in the Composable Fine-Tuning Algorithm of Large-Scale Models

arXiv: https://arxiv.org/abs/2511.03981 · HTML: https://mlsanity.com/paper/2511.03981/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 25 references that are in the map.
- First posted on arXiv 6 Nov 2025.

## Builds on (most cited first, 11 of 11)

- [Combinatorial optimization and reasoning with graph neural networks](https://mlsanity.com/paper/2102.09544/) (2021)
- [End-to-End Constrained Optimization Learning: A Survey](https://mlsanity.com/paper/2103.16378/) (2021)
- [MolCA: Molecular Graph-Language Modeling with Cross-Modal Projector and Uni-Modal Adapter](https://mlsanity.com/paper/2310.12798/) (2023)
- [Modular Deep Learning](https://mlsanity.com/paper/2302.11529/) (2023)
- [GraphEdit: Large Language Models for Graph Structure Learning](https://mlsanity.com/paper/2402.15183/) (2024)
- [GPS++: An Optimised Hybrid MPNN/Transformer for Molecular Property Prediction](https://mlsanity.com/paper/2212.02229/) (2022)
- [Dynamic Prompt Fusion for Multi-Task and Cross-Domain Adaptation in LLMs](https://mlsanity.com/paper/2509.18113/) (2025)
- [Contrastive Learning-Based Dependency Modeling for Anomaly Detection in Cloud Services](https://mlsanity.com/paper/2510.13368/) (2025)
- [Integrating Structure-Aware Attention and Knowledge Graphs in Explainable Recommendation Systems](https://mlsanity.com/paper/2510.10109/) (2025)
- [An Experimental Study of the Transferability of Spectral Graph Networks](https://mlsanity.com/paper/2012.10258/) (2020)
- [Controllable Abstraction in Summary Generation for Large Language Models via Prompt Engineering](https://mlsanity.com/paper/2510.15436/) (2025)

## Built on by (most cited first, 4 of 4)

- [Parameter-Efficient Fine-Tuning with Differential Privacy for Robust Instruction Adaptation in Large Language Models](https://mlsanity.com/paper/2512.06711/) (2025)
- [Deep Q-Learning-Based Intelligent Scheduling for ETL Optimization in Heterogeneous Data Environments](https://mlsanity.com/paper/2512.13060/) (2025)
- [Advancing Text Classification with Large Language Models and Neural Attention Mechanisms](https://mlsanity.com/paper/2512.09444/) (2025)
- [Coordinated Semantic Alignment and Evidence Constraints for Retrieval-Augmented Generation with Large Language Models](https://mlsanity.com/paper/2603.04647/) (2026)

## Abstract (arXiv)

This paper proposes a composable fine-tuning method that integrates graph structural priors with modular adapters to address the high computational cost and structural instability faced by large-scale pre-trained models in multi-task adaptation. The method introduces a relation matrix to model dependencies among tasks, explicitly encoding correlations between nodes and paths into graph structural priors, which provide unified structural constraints for adapter weight allocation and path selection. Modular adapters are embedded into different layers through low-rank mapping and a pluggable mechanism, enabling efficient cross-task composition and reuse under prior guidance. This mechanism not only improves parameter efficiency and training stability but also alleviates path conflicts and redundant computation in multi-task scenarios. Furthermore, experiments on hyperparameter sensitivity, environmental sensitivity, and data sensitivity are conducted to systematically analyze key factors such as routing temperature, gating thresholds, and relation matrix regularization strength, verifying the consistency and superior performance of the method under structural constraints. The results demonstrate that the proposed framework significantly enhances task prediction accuracy, adapter weight allocation precision, and overall computational efficiency while maintaining model lightweight design, highlighting the synergistic advantages of graph priors and modular mechanisms in composable fine-tuning.