Targeted Interpretable Safety Neuron Enhancement for Multilingual Vision-Language Large Models

With the widespread deployment of vision-language large models (VLLMs), their safety alignment faces dual challenges across languages and modalities. Existing methods model multilingual and multimodal safety separately, overlooking coupled risks between low-resource-language instructions and visual contexts, which hinders the detection of cross-lingual and cross-modal harmful intent and the formation of robust safety boundaries. To address this, we propose a neuron-level interpretable safety alignment framework that identifies safety neurons and performs neuron-targeted safety tuning to jointly mitigate multilingual and multimodal risks. Specifically, we compare FFN representations elicited by harmful requests and benign inputs to identify neuron activation strengths associated with safety refusals. Next, we jointly model neuron activations and corresponding down-projection columns to derive neuron-level saliency, separating general multilingual and multimodal neurons from safety neurons responsible for model defense. Finally, neuron-targeted gradient masking restricts parameter updates to the safety subspace spanned by the identified neurons, enabling precise and interpretable safety enhancement. Extensive experiments show that our method enhances multilingual and multimodal safety by tuning only a few safety neurons, while preserving general capabilities.