mlsanityData to 7 Oct 2026

Contrastive Knowledge Transfer and Robust Optimization for Secure Alignment of Large Language Models

arXiv 2510.27077 ↗First posted 31 Oct 2025cs.CLcited by 3

On the map: LLM alignment / RLHF / preference optimisation → Safety alignment and jailbreak attacks

also in 1 more
Abstract (arXiv)

This paper addresses the limitations of large-scale language models in safety alignment and robustness by proposing a fine-tuning method that combines contrastive distillation with noise-robust training. The method freezes the backbone model and transfers the knowledge boundaries of the teacher model to the student model through distillation, thereby improving semantic consistency and alignment accuracy. At the same time, noise perturbations and robust optimization constraints are introduced during training to ensure that the model maintains stable predictive outputs under noisy and uncertain inputs. The overall framework consists of distillation loss, robustness loss, and a regularization term, forming a unified optimization objective that balances alignment ability with resistance to interference. To systematically validate its effectiveness, the study designs experiments from multiple perspectives, including distillation weight sensitivity, stability analysis under computation budgets and mixed-precision environments, and the impact of data noise and distribution shifts on model performance. Results show that the method significantly outperforms existing baselines in knowledge transfer, robustness, and overall safety, achieving the best performance across several key metrics. This work not only enriches the theoretical system of parameter-efficient fine-tuning but also provides a new solution for building safer and more trustworthy alignment mechanisms.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Safety alignment and jailbreak attacks, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 13 of its 26 references that are in the map. First posted on arXiv 31 Oct 2025.

Builds on

13 of 26 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025PRM-Free Security Alignment of Large Models via Red Teaming and Adversarial Training14 Jul
  2. 2025ARMOR: Aligning Secure and Safe Large Language Models via Meticulous Reasoning14 Jul

Other references · 11

  1. 2021DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharingcited by 1,219
  2. 2021Towards a Unified View of Parameter-Efficient Transfer Learningcited by 863
  3. 2022UL2: Unifying Language Learning Paradigmscited by 292
  4. 2023Parameter-Efficient Fine-Tuning Design Spacescited by 67
  5. 2021Complementary Relation Contrastive Distillationcited by 42
  6. 2021Faster Meta Update Strategy for Noise-Robust Deep Learningcited by 38
  7. 2021XtremeDistilTransformers: Task Transfer for Task-agnostic Distillationcited by 18
  8. 2025Trustworthy Summarization via Uncertainty Quantification and Risk Awareness in Large Language Modelscited by 10
  9. 2025Structure-Learnable Adapter Fine-Tuning for Parameter-Efficient Large Language Modelscited by 9
  10. 2025Dynamic Prompt Fusion for Multi-Task and Cross-Domain Adaptation in LLMscited by 9
  11. 2025High Fidelity Text to Image Generation with Contrastive Alignment and Structural Guidancecited by 6

Built on by

Cited by year

Citing papers per year: 2025: 3.

3 papers.

  1. 2025Parameter-Efficient Fine-Tuning with Differential Privacy for Robust Instruction Adaptation in Large Language Modelscited by 5
  2. 2025Structure-Aware Decoding Mechanisms for Complex Entity Extraction with Large-Scale Language Modelscited by 3
  3. 2025Advancing Text Classification with Large Language Models and Neural Attention Mechanismscited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2026Aligned but Fragile: Enhancing LLM Safety Robustness via Zeroth-Order Optimization
  2. 2024Decoupled Alignment for Robust Plug-and-Play Adaptation
  3. 2024Locking Down the Finetuned LLMs Safety
  4. 2025Alignment and Safety in Large Language Models: Safety Mechanisms, Training Paradigms, and Emerging Challenges
  5. 2024Antidote: Post-fine-tuning Safety Alignment for Large Language Models against Harmful Fine-tuning
  6. 2026Few Tokens, Big Leverage: Preserving Safety Alignment by Constraining Safety Tokens during Fine-tuning
  7. 2025SafeMERGE: Preserving Safety Alignment in Fine-Tuned Large Language Models via Selective Layer-Wise Model Merging
  8. 2026Distilling What Matters: Confidence-Aware Selective Distillation for Large Language Models