mlsanityData to 14 Oct 2026

GGSS: Geodesic-Gated Spherical Steering for Inference-Time Debiasing of Generative Vision-Language Models

arXiv 2608.25375 ↗First posted 26 Aug 2026cs.CY cs.CL cs.CVcited by 0

On the map: Multimodal LLMs / vision-language models → Social bias and fairness in VLMs

Abstract (arXiv)

Generative vision-language models (VLMs) are increasingly used in human-centered settings, yet they can produce demographically biased outputs even when images differ only in controlled attributes such as perceived race or gender. However, existing inference-time debiasers were largely designed for static embeddings or CLIP-like models rather than generative VLMs. We propose GGSS---Geodesic-Gated Spherical Steering---a norm-preserving intervention that discovers a counterfactual bias subspace on the unit hypersphere, steers visual tokens along geodesic arcs, and uses an adaptive gate to focus correction on tokens that carry stronger demographic signal. We evaluate four generative VLMs against ten adapted inference-time debiasing baselines and prompt-based mitigation under a single operating-point protocol across categorical, pairwise, and occupation-gender bias tests, while also measuring general visual-language capability. GGSS achieves the lowest average bias on all four models, significant on three of four backbones under paired permutation tests, while preserving MMStar accuracy within +/- 0.6 p.p. of the unsteered baseline. Code is available at https://github.com/dukesun99/GGSS.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Social bias and fairness in VLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 28 of its 52 references that are in the map. First posted on arXiv 26 Aug 2026.

Builds on

28 of 52 references.

Closest ancestors · 10

Same direction, newest first

  1. 2026Interpretable Debiasing of Vision-Language Models for Social Fairness27 Feb
  2. 2026Measuring Social Bias in Vision-Language Models with Face-Only Counterfactuals from Real Photos11 Jan
  3. 2025PRISM: Reducing Spurious Implicit Biases in Vision-Language Models with LLM-Guided Embedding Projection11 Jul
  4. 2025Biases Propagate in Encoder-based Vision-Language Models: A Systematic Analysis From Intrinsic Measures to Zero-shot Retrieval Outcomes6 Jun
  5. 2024Debiasing Large Vision-Language Models by Ablating Protected Attribute Representations17 Oct
  6. 2024A Unified Debiasing Approach for Vision-Language Models across Modalities and Tasks10 Oct
  7. 2024Social Debiasing for Fair Multi-modal LLMs13 Aug
  8. 2023SocialCounterfactuals: Probing and Mitigating Intersectional Social Biases in Vision-Language Models with Counterfactual Examples30 Nov
  9. 2023DeAR: Debiasing Vision-Language Models with Additive Residuals18 Mar
  10. 2023Debiasing Vision-Language Models via Biased Prompts31 Jan

Other references · 18

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,420
  3. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,336
  4. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,302
  5. 2020Null It Out: Guarding Protected Attributes by Iterative Nullspace Projectioncited by 455
  6. 2024VLMEvalKit: An Open-Source Toolkit for Evaluating Large Multi-Modality Modelscited by 455
  7. 2023LEACE: Perfect linear concept erasure in closed formcited by 267
  8. 2019Black is to Criminal as Caucasian is to Police: Detecting and Removing Multiclass Bias in Word Embeddingscited by 147
  9. 2020Measuring Social Biases in Grounded Vision and Language Embeddingscited by 61
  10. 2025AlphaSteer: Learning Refusal Steering with Principled Null-Space Constraintcited by 49
  11. 2024Joint Vision-Language Social Bias Removal for CLIPcited by 18
  12. 2024Growth Inhibitors for Suppressing Inappropriate Image Concepts in Diffusion Modelscited by 12
6 more
  1. 2025Responsible Diffusion Models via Constraining Text Embeddings within Safe Regionscited by 8
  2. 2025Don't Reinvent the Wheel: Efficient Instruction-Following Text Embedding based on Guided Space Transformationcited by 7
  3. 2025PRISM: A Framework for Producing Interpretable Political Bias Embeddings with Political-Aware Cross-Encodercited by 5
  4. 2025One Swallow Does Not Make a Summer: Understanding Semantic Structures in Embedding Spacescited by 4
  5. 2026Do Prompts Guarantee Safety? Mitigating Toxicity from LLM Generations through Subspace Interventioncited by 1
  6. 2026Sparse Autoencoders as Plug-and-Play Firewalls for Adversarial Attack Detection in VLMscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Bias Is a Subspace, Not a Coordinate: A Geometric Rethinking of Post-hoc Debiasing in Vision-Language Models
  2. 2025GS-Bias: Global-Spatial Bias Learner for Single-Image Test-Time Adaptation of Vision-Language Models
  3. 2024debiaSAE: Benchmarking and Mitigating Vision-Language Model Bias
  4. 2026When Language Overwrites Vision: Over-Alignment and Geometric Debiasing in Vision-Language Models
  5. 2025Pathological Truth Bias in Vision-Language Models
  6. 2024BendVLM: Test-Time Debiasing of Vision-Language Embeddings
  7. 2025GRAID: Enhancing Spatial Reasoning of VLMs Through High-Fidelity Data Generation
  8. 2026ViD: Vision-Dominant Gender Bias Mitigation for Large Vision-Language Models