# Dynamic Prompt Fusion for Multi-Task and Cross-Domain Adaptation in LLMs

arXiv: https://arxiv.org/abs/2509.18113 · HTML: https://mlsanity.com/paper/2509.18113/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 25 references that are in the map.
- First posted on arXiv 9 Sep 2025.

## Builds on (most cited first, 11 of 11)

- [Multitask Prompted Training Enables Zero-Shot Task Generalization](https://mlsanity.com/paper/2110.08207/) (2021)
- [SPoT: Better Frozen Model Adaptation through Soft Prompt Transfer](https://mlsanity.com/paper/2110.07904/) (2021)
- [Multitask Prompt Tuning Enables Parameter-Efficient Transfer Learning](https://mlsanity.com/paper/2303.02861/) (2023)
- [MmAP : Multi-modal Alignment Prompt for Cross-domain Multi-task Learning](https://mlsanity.com/paper/2312.08636/) (2023)
- [DiPrompT: Disentangled Prompt Tuning for Multiple Latent Domain Generalization in Federated Learning](https://mlsanity.com/paper/2403.08506/) (2024)
- [Task Prompt Vectors: Effective Initialization through Multi-Task Soft-Prompt Transfer](https://mlsanity.com/paper/2408.01119/) (2024)
- [Multitask Pre-training of Modular Prompt for Chinese Few-Shot Learning](https://mlsanity.com/paper/2210.07565/) (2022)
- [Collaborative Multi-Agent Reinforcement Learning Approach for Elastic Cloud Resource Scaling](https://mlsanity.com/paper/2507.00550/) (2025)
- [Structured Memory Mechanisms for Stable Context Representation in Large Language Models](https://mlsanity.com/paper/2505.22921/) (2025)
- [Clinical NLP with Attention-Based Deep Learning for Multi-Disease Prediction](https://mlsanity.com/paper/2507.01437/) (2025)
- [Enhancing Few-Shot Transfer Learning with Optimized Multi-Task Prompt Tuning through Modular Prompt Composition](https://mlsanity.com/paper/2408.13227/) (2024)

## Built on by (most cited first, 9 of 9)

- [Modular Task Decomposition and Dynamic Collaboration in Multi-Agent Systems Driven by Large Language Models](https://mlsanity.com/paper/2511.01149/) (2025)
- [Deep Learning Approach for Clinical Risk Identification Using Transformer Modeling of Heterogeneous EHR Data](https://mlsanity.com/paper/2511.04158/) (2025)
- [Structural Priors and Modular Adapters in the Composable Fine-Tuning Algorithm of Large-Scale Models](https://mlsanity.com/paper/2511.03981/) (2025)
- [Contrastive Knowledge Transfer and Robust Optimization for Secure Alignment of Large Language Models](https://mlsanity.com/paper/2510.27077/) (2025)
- [Multi-Scale Feature Fusion and Graph Neural Network Integration for Text Classification with Large Language Models](https://mlsanity.com/paper/2511.05752/) (2025)
- [Machine Learning Approaches to Clinical Risk Prediction: Multi-Scale Temporal Alignment in Electronic Health Records](https://mlsanity.com/paper/2511.21561/) (2025)
- [Dynamic Anomaly Identification in Accounting Transactions via Multi-Head Self-Attention Networks](https://mlsanity.com/paper/2511.12122/) (2025)
- [CaliCausalRank: Calibrated Multi-Objective Ad Ranking with Robust Counterfactual Utility Optimization](https://mlsanity.com/paper/2602.18786/) (2026)
- [Structured Prompt Optimization for Few-Shot Text Classification via Semantic Alignment in Latent Space](https://mlsanity.com/paper/2602.23753/) (2026)

## Abstract (arXiv)

This study addresses the generalization limitations commonly observed in large language models under multi-task and cross-domain settings. Unlike prior methods such as SPoT, which depends on fixed prompt templates, our study introduces a unified multi-task learning framework with dynamic prompt scheduling mechanism. By introducing a prompt pool and a task-aware scheduling strategy, the method dynamically combines and aligns prompts for different tasks. This enhances the model's ability to capture semantic differences across tasks. During prompt fusion, the model uses task embeddings and a gating mechanism to finely control the prompt signals. This ensures alignment between prompt content and task-specific demands. At the same time, it builds flexible sharing pathways across tasks. In addition, the proposed optimization objective centers on joint multi-task learning. It incorporates an automatic learning strategy for scheduling weights, which effectively mitigates task interference and negative transfer. To evaluate the effectiveness of the method, a series of sensitivity experiments were conducted. These experiments examined the impact of prompt temperature parameters and task number variation. The results confirm the advantages of the proposed mechanism in maintaining model stability and enhancing transferability. Experimental findings show that the prompt scheduling method significantly improves performance on a range of language understanding and knowledge reasoning tasks. These results fully demonstrate its applicability and effectiveness in unified multi-task modeling and cross-domain adaptation.