mlsanityData to 14 Oct 2026

3D FaceShell: Attribute Transfer in 3D Face Avatars as a VLM Defense Mechanism

arXiv 2607.16280 ↗First posted 9 Jul 2026cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Photorealistic 3D face avatars are increasingly deployed as reusable digital assets across applications such as telepresence, animation, and personalized media. At the same time, vision-language models (VLMs) can infer sensitive attributes from rendered images with open-ended semantic reasoning without any fine-tuning. This creates a new privacy challenge: once a 3D face avatar is shared, any of its renderings can be analyzed to extract high-level facial attributes. Existing defenses largely operate in 2D image space and do not address identity-preserving semantic manipulation of 3D facial representations. We propose 3D FaceShell, a framework for steering VLM interpretations of faces rendered from 3D models while preserving geometric fidelity and facial identity. 3D FaceShell augments the original 3D representation with a learnable Gaussian shell that produces subtle, spatially distributed perturbations optimized through multi-view embedding alignment. The perturbations are designed to be visually inconspicuous yet sufficient to redirect VLM-based attribute inference in a view-consistent manner. Extensive experiments on reconstructed celebrity face avatars and multiple black-box VLMs demonstrate that 3D FaceShell significantly increases attribute injection and mismatch rates while maintaining high perceptual similarity and identity consistency. Our results show that it is possible to manipulate VLM-level semantic interpretation of 3D faces without compromising their human-recognizable appearance.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 41 of its 48 references that are in the map. First posted on arXiv 9 Jul 2026.

Builds on

41 of 48 references.

References · 41

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2024The Llama 3 Herd of Modelscited by 14,715
  4. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  5. 2023Visual Instruction Tuningcited by 7,869
  6. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,226
  7. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  8. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,109
  9. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
  10. 2018OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fieldscited by 1,162
  11. 20234D Gaussian Splatting for Real-Time Dynamic Scene Renderingcited by 1,157
  12. 2024LGM: Large Multi-View Gaussian Model for High-Resolution 3D Content Creationcited by 739
29 more
  1. 2023Mip-Splatting: Alias-free 3D Gaussian Splattingcited by 667
  2. 2024LLaVA-NeXT-Interleave: Tackling Multi-image, Video, and 3D in Large Multimodal Modelscited by 584
  3. 2025VideoLLaMA 3: Frontier Multimodal Foundation Models for Image and Video Understandingcited by 518
  4. 2023LightGaussian: Unbounded 3D Gaussian Compression with 15x Reduction and 200+ FPScited by 436
  5. 2023Visual Adversarial Examples Jailbreak Aligned Large Language Modelscited by 328
  6. 2023On Evaluating Adversarial Robustness of Large Vision-Language Modelscited by 284
  7. 2023GaussianAvatars: Photorealistic Head Avatars with Rigged 3D Gaussianscited by 260
  8. 2023How Robust is Google's Bard to Adversarial Image Attacks?cited by 183
  9. 2018Generating 3D Adversarial Point Cloudscited by 144
  10. 2024Jailbreaking Attack against Multimodal Large Language Modelcited by 135
  11. 2024CAP4D: Creating Animatable 4D Portrait Avatars with Morphable Multi-View Diffusion Modelscited by 52
  12. 2024Efficient Generation of Targeted and Transferable Adversarial Examples for Vision-Language Models Via Diffusion Modelscited by 38
  13. 2024Private Attribute Inference from Images with Vision-Language Modelscited by 37
  14. 2025A Frustratingly Simple Yet Highly Effective Attack Baseline: Over 90% Success Rate Against the Strong Black-box Models of GPT-4.5/4o/o1cited by 37
  15. 2023Towards Effective Adversarial Textured 3D Meshes on Physical Face Recognitioncited by 33
  16. 2025HRAvatar: High-Quality and Relightable Gaussian Head Avatarcited by 25
  17. 2025FaceXBench: Evaluating Multimodal LLMs on Face Understandingcited by 24
  18. 2022Physical-World Optical Adversarial Attacks on 3D Face Recognitioncited by 23
  19. 2024FaceLift: Learning Generalizable Single Image 3D Face Reconstruction from Synthetic Headscited by 22
  20. 2024Gaussian Splatting Under Attack: Investigating Adversarial Noise in 3D Objectscited by 11
  21. 2025Face-LLaVA: Facial Expression and Attribute Understanding through Instruction Tuningcited by 11
  22. 2023Targeted Adversarial Attacks on Generalizable Neural Radiance Fieldscited by 10
  23. 2025V-Attack: Targeting Disentangled Value Features for Controllable Adversarial Attacks on LVLMscited by 9
  24. 20253D Gaussian Splatting Driven Multi-View Robust Physical Adversarial Camouflage Generationcited by 7
  25. 2023Adversarial Attacks and Defenses on 3D Point Cloud Classification: A Surveycited by 5
  26. 2025Towards Robust Multimodal Large Language Models Against Jailbreak Attackscited by 5
  27. 2024Adversarial Attacks on Both Face Recognition and Face Anti-spoofing Modelscited by 4
  28. 2024A Survey on Physical Adversarial Attacks against Face Recognition Systemscited by 4
  29. 2024Exploring Vision Language Models for Facial Attribute Recognition: Emotion, Race, Gender, and Agecited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024FaceGPT: Self-supervised Learning to Chat about 3D Human Faces
  2. 2024ID-to-3D: Expressive ID-guided 3D Heads via Score Distillation Sampling
  3. 2025Who Can See Through You? Adversarial Shielding Against VLM-Based Attribute Inference Attacks
  4. 2026Defending Wearable VLMs Against Private Attribute Inference
  5. 2025Disentangling 3D from Large Vision-Language Models for Controlled Portrait Generation
  6. 2026Splatshot: 3D Face Avatar Generation from a Single Unconstrained Photo
  7. 2026DP^2-VL: Private Photo Dataset Protection by Data Poisoning for Vision-Language Models
  8. 2025HairCUP: Hair Compositional Universal Prior for 3D Gaussian Avatars