Metric-Fair Prompting: Treating Similar Samples Similarly

AI summary

Overview

Research area: Natural Language Processing / fairness-aware prompting for large language models, applied to medical multiple-choice question answering.

Technical level: Intermediate. The paper pairs prompt engineering with a formal individual-fairness formulation (a Lipschitz-style constraint on a score function), so readers benefit from familiarity with metric fairness, classifier margins, and text embeddings, though the prompting protocol itself is described concretely.

Scope: The paper proposes Metric-Fair Prompting, a joint-inference prompting protocol that solves pairs of similar medical questions together under a Lipschitz-like fairness constraint, and reports accuracy on the MedQA (US) test split.

What This Paper Is About

Standard prompting asks an LLM to answer each multiple-choice question in isolation, which can produce inconsistent answers for questions that are clinically nearly identical. The authors treat each (question, option) pair as a binary instance (+1 for correct, −1 for incorrect) and enforce the individual-fairness principle that similar instances should receive similar scores and therefore similar decisions. Their goal is to improve both consistency and accuracy on high-stakes clinical multiple-choice questions by having the model reason about two similar questions jointly rather than separately.

Key Contributions

1. A fairness-aware prompting framework that casts MedQA as binary classification over (question, option) pairs and enforces a metric-based Lipschitz constraint to encourage individual fairness.
2. A joint-inference protocol that feeds pairs of similar questions to the LLM, enabling cross-item consistency and reducing near-boundary errors.
3. An empirical demonstration on MedQA (US) that the protocol improves accuracy over single-item prompting (68% to 84% with Qwen3-14B), showing that fairness-guided, confidence-oriented reasoning can raise LLM accuracy in clinical multiple-choice settings.

Main Findings

• Accuracy gain from joint, fairness-constrained prompting: On the MedQA (US) test split, Metric-Fair Prompting with Qwen3-14B improves accuracy from 68% (single-item prompting) to 84% (two-item, metric-fair, joint inference). Table 3 reports the same values as 68.0 versus 84.0.
• Similar questions do appear in the dataset: The three most similar question pairs selected by Qwen3-4B embedding cosine similarity scored 0.9612, 0.9020, and 0.8314.
• Consistency on near-duplicate clinical stems (cosine = 0.9612): Two stems that are clinically indistinguishable across symptoms, exam, labs, and biopsy, differing only in age, were both mapped to the same correct option, "Adverse effect of anesthetic." The authors read this as individual fairness, demographic robustness (age ignored when not clinically determinative), and boundary stability.
• Shared evidence, different questions (cosine = 0.9020): Both stems referenced the same study abstract, one asking about interpretation of the standard error (sample size and variability) and the other about the statistical method for group differences. Joint presentation let the model build one internal representation of the study structure and project each item to the appropriate decision subspace.
• Mechanism-level coherence (cosine = 0.8314): Two trauma cases with a shared mechanism (unrestrained MVA) and overlapping findings both pointed to Cardiac contusion (Option A), consistent with elevated troponins or LV dysfunction and absence of signs of MI or dissection.
• Pairwise setting acts as a regularizer: The authors argue that when two similar items are solved jointly, option choices that are inconsistent across the pair are penalized by the metric constraint, improving reliability near the decision boundary.
• Reported instability: The limitations section states that despite the observed gains, results are "not always stable," and that performance is not completely understood from a rigorous theoretical perspective.
• Error bars and confidence intervals are not reported in the main text: The paper presents single accuracy values per setting. The NeurIPS checklist item on statistical significance is marked [Yes] but carries only a placeholder justification in the provided content.

Methodology in Plain English

Each answer choice combined with its question is treated as a yes/no item: correct or incorrect. A score f(x) is assigned to each item, and the size of that score plays the role of confidence, similar to the margin in a support vector machine. Predictions come from a threshold rule, y = 1{f(x) > α}, with α = 1/2 used as the logistic-regression example.

The fairness requirement is a Lipschitz condition: if two inputs are close under a task-relevant metric d (bounded in [0,1]), their scores must stay close, D(f(x), f(x′)) ≤ d(x, x′). The stated objective is to minimize expected loss subject to this constraint for all x, x′ in the input space.

The practical pipeline has five steps:

1. Pair selection. Embed every question stem with Qwen3-4B embedding, compute cosine similarity, and pair each question with its nearest neighbor (excluding itself) to form a two-item batch. This produces N = 1,273 pairs, and a question can appear in more than one pair as a neighbor.
2. Metric fairness. Instruct the LLM that similar items should receive similar decisions, and that decisions must rest on clinically determinative features rather than sensitive attributes such as age, gender, or race unless those are directly relevant.
3. Margin/half-space reasoning. For each (question, option), compute the score f(x), eliminate clear negatives with large negative margins, and resolve near-boundary candidates using decisive clinical discriminators such as guidelines, pathognomonic findings, and contraindications. The selected option for each question is the one with the largest positive margin.
4. Cross-item consistency. Reconcile near ties within the pair, preferring choices that stay consistent across similar items.
5. Strict output. Emit only machine-parsable JSON, e.g. {"index": i, "answer": "A|B|C|D|E"} for i in {1, 2}.

Because a question may appear in several pairs and receive two predictions, the authors use a conflict-resolution step: when predictions disagree, a light-weight review prompt asks the model to re-evaluate both items jointly and output an answer with a scalar confidence. The higher-confidence answer is kept; if confidences tie, the answer with the larger decision margin is preferred (when available), with a fallback to the original single-item prediction.

Experimental setup: All experiments ran on a single NVIDIA RTX 6000 Ada with no multi-GPU or model parallelism. The authors evaluate Qwen models from HuggingFace via transformers with Unsloth optimizations for efficient inference, using a low temperature of T = 0.2 and greedy decoding (do_sample=False) to stabilize multiple-choice predictions. The baseline is single-item prompting with standard instructions, one question at a time, and the reported metric is accuracy (%) on the MedQA (US) test split of 1,273 items with one correct option (A–E). Table 1 illustrates the premise with two similar patient vignettes whose deterministic features point to methotrexate-related folate pathway toxicity, with prevention via folate supplementation or folinic acid (leucovorin) rescue.

Why This Matters

Impact on research. The paper connects individual fairness, typically studied in settings such as bandit problems, metric-fair active learning of homogeneous halfspaces, and PAC learning, to prompt engineering for LLMs. It contrasts with chain-of-thought, self-consistency, Tree-of-Thoughts, and ReAct, which the authors characterize as treating items independently and optimizing intra-item reasoning, by instead introducing an inter-item coupling through a similarity metric.

Real-world applications:

• Clinical decision support, where clinically equivalent cases should not receive divergent recommendations due to incidental demographic or stylistic differences.
• Medical education and licensing-style exam question answering, the direct evaluation setting of this paper.
• Fairness auditing and red-teaming of LLM systems deployed in high-stakes domains, using consistency across similar inputs as a measurable signal.
• Medical triage or information retrieval, where grouping similar cases jointly can reduce near-boundary errors in evidence interpretation.

Industry relevance. The findings matter to healthcare AI developers, model providers, and regulated industries that need auditable consistency properties rather than accuracy alone. The protocol requires no model retraining, only embedding-based pairing and a prompt template, which keeps adoption cost low; the requirement of strict JSON output also makes it directly compatible with automated evaluation harnesses.

Future Directions

1. Improve stability via calibrated decoding, temperature-free beam search, and ensembling, since the authors report that results are not always stable.
2. Learn task-specific similarity metrics with clinical supervision, rather than relying on general-purpose sentence embeddings.
3. Integrate confidence estimation and selective answering, so the system can abstain or defer when margins are small.
4. Explore pair construction beyond nearest neighbors, such as cluster-then-cover or active pairing, and extend evaluation to broader datasets and multilingual settings.
5. Add human-in-the-loop review and bias audits to strengthen reliability, fairness, and generalizability while keeping the method practical and transparent.

Target Audience

Researchers working on fairness in machine learning and LLM alignment; NLP practitioners building prompting or inference-time reasoning methods; clinical informatics and healthcare AI teams evaluating models on medical question answering; and anyone interested in connecting formal individual-fairness constraints directly to prompt design rather than to model training.

Authors’ abstract

We introduce \emph{Metric-Fair Prompting}, a fairness-aware prompting framework that guides large language models (LLMs) to make decisions under metric-fairness constraints. In the application of multiple-choice medical question answering, each {(question, option)} pair is treated as a binary instance with label $+1$ (correct) or $-1$ (incorrect). To promote {individual fairness}~--~treating similar instances similarly~--~we compute question similarity using NLP embeddings and solve items in \emph{joint pairs of similar questions} rather than in isolation. The prompt enforces a global decision protocol: extract decisive clinical features, map each \((\text{question}, \text{option})\) to a score $f(x)$ that acts as confidence, and impose a Lipschitz-style constraint so that similar inputs receive similar scores and, hence, consistent outputs. Evaluated on the {MedQA (US)} benchmark, Metric-Fair Prompting is shown to improve performance over standard single-item prompting, demonstrating that fairness-guided, confidence-oriented reasoning can enhance LLM accuracy on high-stakes clinical multiple-choice questions.