Behavioral Reprogramming of Open-Weights Models: Cognitive Plasticity and Alignment Bounds
On the map: LLM alignment / RLHF / preference optimisation → LLM preference optimization methods
Abstract (arXiv)
Large language models (LLMs) are predominantly aligned to function as passive, sycophantic assistants. We challenge this default paradigm by empirically evaluating the cognitive plasticity of open-weight architectures when subjected to rigorous behavioral reprogramming. Our objective is to induce a proactive, Socratic conversational framework, characterized by high-frequency question generation under strictly constrained high-performance computing (HPC) conditions. Through a massively parallelized hyperparameter sweep comprising 405 HPC jobs, we define precise mathematical bounds for parameter-efficient fine-tuning (PEFT). We identify an architectural threshold at LoRA rank r=16 and demonstrate via extensive epoch ablation that generalization capacity strictly reaches its optimal convergence within an optimized training window of e ∈ [2, 3] depending on dataset density (minimum validation loss of 0.919). Furthermore, scaling model capacity to 14B parameters yielded a lower localized evaluation perplexity (1.414). Subsequent Direct Preference Optimization (DPO) successfully decoupled the underlying assertive behavior from localized syntax, while rigorous cross-lingual stress testing reveals both the capabilities and the structural boundaries of zero-shot persona transfer, demonstrating robust alignment in closely related linguistic families alongside identifiable degradation pathways in morphologically distant targets. These findings establish a rigorous empirical framework for compute-efficient, cross-lingual behavioral modification.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in LLM preference optimization methods, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 32 of its 41 references that are in the map. First posted on arXiv 13 Aug 2026.
Builds on
32 of 41 references.
Closest ancestors · 2
Same direction, newest first
- 2022Training language models to follow instructions with human feedback4 Mar
- 2021A General Language Assistant as a Laboratory for Alignment1 Dec
Other references · 30
- 2017Attention Is All You Needcited by 60,361
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2023GPT-4 Technical Reportcited by 18,396
- 2024The Llama 3 Herd of Modelscited by 14,715
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
- 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
- 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
- 2025Qwen3 Technical Reportcited by 8,820
- 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
- 2020Measuring Massive Multitask Language Understandingcited by 7,963
- 2020Scaling Laws for Neural Language Modelscited by 6,586
18 more
- 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
- 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
- 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,915
- 2017Deep reinforcement learning from human preferencescited by 3,912
- 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,795
- 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
- 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
- 2023Mistral 7Bcited by 3,164
- 2022Training Compute-Optimal Large Language Modelscited by 3,019
- 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,592
- 2020Learning to summarize from human feedbackcited by 2,588
- 2023A Survey of Large Language Modelscited by 2,576
- 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,234
- 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,998
- 2022Red Teaming Language Models with Language Modelscited by 1,026
- 2020What is the State of Neural Network Pruning?cited by 583
- 2023DeepSpeed-Chat: Easy, Fast and Affordable RLHF Training of ChatGPT-like Models at All Scalescited by 108
- 2023InstructZero: Efficient Instruction Optimization for Black-Box Large Language Modelscited by 68
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Revealing Behavioral Plasticity in Large Language Models: A Token-Conditional Perspective
- 2025Towards Alignment-Centric Paradigm: A Survey of Instruction Tuning in Large Language Models
- 2024Insights into Alignment: Evaluating DPO and its Variants Across Multiple Tasks
- 2023OpenAssistant Conversations -- Democratizing Large Language Model Alignment
- 2025PromptCoT 2.0: Scaling Prompt Synthesis for Large Language Model Reasoning
- 2024Improving the Robustness of Large Language Models via Consistency Alignment
- 2025A Survey on Post-training of Large Language Models
- 2026SeDT: Sentence-Transformer Decision-Transformer Conditioning for Multi-Turn Conversation Reliability