# Chain-of-Thought Prompting for Demographic Inference with Large Multimodal Models

arXiv: https://arxiv.org/abs/2405.15687 · HTML: https://mlsanity.com/paper/2405.15687/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Multimodal reasoning, one of 17 directions in LLM reasoning.
- Builds on 15 of its 20 references that are in the map.
- First posted on arXiv 24 May 2024.

## Placed in

- LLM reasoning / [Multimodal reasoning](https://mlsanity.com/lr/multimodal-reasoning/)
- Multimodal LLMs / vision-language models / [MLLM training and visual instruction tuning](https://mlsanity.com/mllm/mllm-training-and-visual-instruction-tuning/)

## Builds on (most cited first, 15 of 15)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [Improved Baselines with Visual Instruction Tuning](https://mlsanity.com/paper/2310.03744/) (2023)
- [InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning](https://mlsanity.com/paper/2305.06500/) (2023)
- [MiniGPT-v2: large language model as a unified interface for vision-language multi-task learning](https://mlsanity.com/paper/2310.09478/) (2023)
- [Age Progression/Regression by Conditional Adversarial Autoencoder](https://mlsanity.com/paper/1702.08423/) (2017)
- [InternLM-XComposer: A Vision-Language Large Model for Advanced Text-image Comprehension and Composition](https://mlsanity.com/paper/2309.15112/) (2023)
- [Rank consistent ordinal regression for neural networks with application to age estimation](https://mlsanity.com/paper/1901.07884/) (2019)
- [CoCoT: Contrastive Chain-of-Thought Prompting for Large Multimodal Models with Multiple Image Inputs](https://mlsanity.com/paper/2401.02582/) (2024)
- [Moving Window Regression: A Novel Approach to Ordinal Regression](https://mlsanity.com/paper/2203.13122/) (2022)
- [MiVOLO: Multi-input Transformer for Age and Gender Estimation](https://mlsanity.com/paper/2307.04616/) (2023)
- [FLAC: Fairness-Aware Representation Learning by Suppressing Attribute-Class Associations](https://mlsanity.com/paper/2304.14252/) (2023)
- [Multi-task Learning of Cascaded CNN for Facial Attribute Classification](https://mlsanity.com/paper/1805.01290/) (2018)
- [Face Off: Polarized Public Opinions on Personal Face Mask Usage during the COVID-19 Pandemic](https://mlsanity.com/paper/2011.00336/) (2020)

## Built on by (most cited first, 4 of 4)

- [Multimodal Chain-of-Thought Reasoning: A Comprehensive Survey](https://mlsanity.com/paper/2503.12605/) (2025)
- [FINECAPTION: Compositional Image Captioning Focusing on Wherever You Want at Any Granularity](https://mlsanity.com/paper/2411.15411/) (2024)
- [MIRA: Multimodal Iterative Reasoning Agent for Image Editing](https://mlsanity.com/paper/2511.21087/) (2025)
- [BiasDora: Exploring Hidden Biased Associations in Vision-Language Models](https://mlsanity.com/paper/2407.02066/) (2024)

## Abstract (arXiv)

Conventional demographic inference methods have predominantly operated under the supervision of accurately labeled data, yet struggle to adapt to shifting social landscapes and diverse cultural contexts, leading to narrow specialization and limited accuracy in applications. Recently, the emergence of large multimodal models (LMMs) has shown transformative potential across various research tasks, such as visual comprehension and description. In this study, we explore the application of LMMs to demographic inference and introduce a benchmark for both quantitative and qualitative evaluation. Our findings indicate that LMMs possess advantages in zero-shot learning, interpretability, and handling uncurated 'in-the-wild' inputs, albeit with a propensity for off-target predictions. To enhance LMM performance and achieve comparability with supervised learning baselines, we propose a Chain-of-Thought augmented prompting approach, which effectively mitigates the off-target prediction issue.