mlsanityData to 14 Oct 2026

When a Zero-Shooter Cheats: Improving Age Estimation via Activation Steering

arXiv 2605.17658 ↗First posted 17 May 2026cs.LGcited by 0

On the map: LLM safety and security → Safety alignment, refusal and guardrails

Abstract (arXiv)

Different age-related regulations have been proposed to protect minors from harmful content and interactions online. Automated age estimation is central to enforcing such regulations, and vision-language models (VLMs) achieve state-of-the-art performance on this task. However, we find that the zero-shot nature of VLM-based age estimation produces an unexpected side effect we call the identity shortcut: Instead of estimating age from visual features, VLMs tend to identify the depicted person and infer their age from memorized knowledge. This phenomenon leads to substantially incorrect predictions when non-celebrities are misidentified as celebrities. It also produces deceptively high robustness to noise and adversarial perturbations on celebrity images, which dominate popular benchmarks. To mitigate this, we propose an activation steering method that suppresses the shortcut by intervening on the hidden states of the VLM. This method improves age estimation accuracy for both memorized and unseen identities, reducing mean absolute error by up to 25% across popular benchmarks.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Safety alignment, refusal and guardrails, one of 12 directions in LLM safety and security. Builds on 22 of its 35 references that are in the map. First posted on arXiv 17 May 2026.

Builds on

22 of 35 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Exploring the Limits of Zero Shot Vision Language Models for Hate Meme Detection: The Vulnerabilities and their Interpretations19 Feb

Other references · 21

  1. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  2. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  3. 2014Deep Learning Face Attributes in the Wildcited by 3,693
  4. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,877
  5. 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,141
  6. 2025Gemma 3 Technical Reportcited by 1,824
  7. 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,652
  8. 2016Joint Face Detection and Alignment using Multi-task Cascaded Convolutional Networkscited by 903
  9. 2023In-Context Learning Creates Task Vectorscited by 312
  10. 2023LAMM: Language-Assisted Multi-Modal Instruction-Tuning Dataset, Framework, and Benchmarkcited by 169
  11. 2023On the Robustness of Large Multimodal Models Against Image Adversarial Attackscited by 92
  12. 2024RoTBench: A Multi-Level Benchmark for Evaluating the Robustness of Large Language Models in Tool Learningcited by 47
9 more
  1. 2023MiVOLO: Multi-input Transformer for Age and Gender Estimationcited by 42
  2. 2025Analysing the Robustness of Vision-Language-Models to Common Corruptionscited by 16
  3. 2024Zero-Shot Visual Reasoning by Vision-Language Models: Benchmarking and Analysiscited by 15
  4. 2025Zero-shot Benchmarking: A Framework for Flexible and Scalable Automatic Evaluation of Language Modelscited by 14
  5. 2025Double Visual Defense: Adversarial Pre-training and Instruction Tuning for Improving Vision-Language Model Robustnesscited by 10
  6. 2024Beyond Specialization: Assessing the Capabilities of MLLMs in Age and Gender Estimationcited by 9
  7. 2026Out of the box age estimation through facial imagery: A Comprehensive Benchmark of Vision-Language Models vs. out-of-the-box Traditional Architecturescited by 5
  8. 2025Evaluating Concept Filtering Defenses against Child Sexual Abuse Material Generation by Text-to-Image Modelscited by 4
  9. 2025Underage Detection through a Multi-Task and MultiAge Approach for Screening Minors in Unconstrained Imagerycited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Can a Teenager Fool an AI? Evaluating Low-Cost Cosmetic Attacks on Age Estimation Systems
  2. 2026VLAgeBench: Benchmarking Large Vision-Language Models for Zero-Shot Human Age Estimation
  3. 2026Face Age Verification Vulnerabilities Under Simple Appearance Manipulations
  4. 2026Frontier vision-language models have overtaken young adults at detecting AI-generated portraits -- but not their calibration
  5. 2025From Cradle to Cane: A Two-Pass Framework for High-Fidelity Lifespan Face Aging
  6. 2025Ageing Drift in Binary Face Templates: A Bits-per-Decade Analysis
  7. 2025DiffClean: Diffusion-based Makeup Removal for Accurate Age Estimation
  8. 2025AgeBooth: Controllable Facial Aging and Rejuvenation via Diffusion Models