# Metric-Fair Prompting: Treating Similar Samples Similarly

arXiv: https://arxiv.org/abs/2512.07608 · HTML: https://mlsanity.com/paper/2512.07608/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 29 references that are in the map.
- First posted on arXiv 8 Dec 2025.

## Builds on (most cited first, 17 of 17)

- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [Self-Refine: Iterative Refinement with Self-Feedback](https://mlsanity.com/paper/2303.17651/) (2023)
- [Qwen Technical Report](https://mlsanity.com/paper/2309.16609/) (2023)
- [Tree of Thoughts: Deliberate Problem Solving with Large Language Models](https://mlsanity.com/paper/2305.10601/) (2023)
- [Equality of Opportunity in Supervised Learning](https://mlsanity.com/paper/1610.02413/) (2016)
- [What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Exams](https://mlsanity.com/paper/2009.13081/) (2020)
- [Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agents](https://mlsanity.com/paper/2201.07207/) (2022)
- [Certifying and removing disparate impact](https://mlsanity.com/paper/1412.3756/) (2014)
- [Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Models](https://mlsanity.com/paper/2305.04091/) (2023)
- [Language Models can Solve Computer Tasks](https://mlsanity.com/paper/2303.17491/) (2023)
- [Active Prompting with Chain-of-Thought for Large Language Models](https://mlsanity.com/paper/2302.12246/) (2023)
- [Compositional Semantic Parsing with Large Language Models](https://mlsanity.com/paper/2209.15003/) (2022)
- [On the Power of Localized Perceptron for Label-Optimal Learning of Halfspaces with Adversarial Noise](https://mlsanity.com/paper/2012.10793/) (2020)
- [Sample-Optimal PAC Learning of Halfspaces with Malicious Noise](https://mlsanity.com/paper/2102.06247/) (2021)
- [A Large-Language Model Framework for Relative Timeline Extraction from PubMed Case Reports](https://mlsanity.com/paper/2504.12350/) (2025)

## Built on by (most cited first, 3 of 3)

- [Immunological Density Shapes Recovery Trajectories in Long COVID](https://mlsanity.com/paper/2601.07854/) (2026)
- [Improving Fairness of Large Language Model-Based ICU Mortality Prediction via Case-Based Prompting](https://mlsanity.com/paper/2512.19735/) (2025)
- [C-kNN-LSH: A Nearest-Neighbor Algorithm for Sequential Counterfactual Inference](https://mlsanity.com/paper/2602.02371/) (2026)

## Abstract (arXiv)

We introduce Metric-Fair Prompting, a fairness-aware prompting framework that guides large language models (LLMs) to make decisions under metric-fairness constraints. In the application of multiple-choice medical question answering, each (question, option) pair is treated as a binary instance with label +1 (correct) or -1 (incorrect). To promote individual fairness~--~treating similar instances similarly~--~we compute question similarity using NLP embeddings and solve items in joint pairs of similar questions rather than in isolation. The prompt enforces a global decision protocol: extract decisive clinical features, map each ((question, option)) to a score f(x) that acts as confidence, and impose a Lipschitz-style constraint so that similar inputs receive similar scores and, hence, consistent outputs. Evaluated on the MedQA (US) benchmark, Metric-Fair Prompting is shown to improve performance over standard single-item prompting, demonstrating that fairness-guided, confidence-oriented reasoning can enhance LLM accuracy on high-stakes clinical multiple-choice questions.