Alternative Fairness and Accuracy Optimization in Criminal Justice

AI summary

Overview

Research area: Algorithmic fairness, with a focus on risk assessment and decision systems in criminal justice. The paper sits at the intersection of machine learning fairness metrics, criminology, and political-science ideas about legitimacy.

Technical level: Intermediate. The paper is largely a conceptual review and position piece, but it includes formal notation (false negative rates, Lipschitz conditions, Wasserstein distance, constrained optimization) alongside legal and policy discussion.

Scope: The paper reviews three families of fairness definitions (group, individual, process), proposes a relaxed group-fairness formulation with a tolerance bound on false negative rate differences, critiques that proposal, and offers a three-pillar deployment framework for public decision systems.

What This Paper Is About

Machine learning fairness has produced a large number of competing definitions, and many of them conflict with one another, which makes them hard to apply in high-stakes settings such as criminal justice. The authors map where group, individual, and process fairness collide, then propose a modified group-fairness objective that replaces exact parity across protected groups with a tolerance bound on differences in false negative rates. Their goal is to make fair solutions more feasible, improve accuracy relative to exact parity, and force the ethical question of error costs into the open.

Key Contributions

1. A structured review of three fairness families. The paper walks through group fairness (demographic parity, equalized odds, equal opportunity, calibration), individual fairness (similar individuals treated similarly), and process fairness (input-side legitimacy through transparency), and it describes the mathematical form of each.

2. A relaxed alternative to standard group fairness. Rather than requiring exactly equal false negative rates across protected groups, the authors minimize a weighted error loss subject to the constraint that false negative rates across groups differ by no more than a tolerance bound, written as |FN(h,v_i) − FN(h,v_j)| ≤ τ.

3. A synthesis of three critiques of group fairness. The authors organize objections into inherent biases in data, latent affirmative action, and the explosion of subgroup constraints, and they argue that these critiques together require value-based judgment rather than technical fixes alone.

4. A three-pillar deployment framework. The paper proposes need-based decisions, transparency and accountability, and narrowly tailored solutions and definitions as guiding principles for agencies using risk assessment tools and similar systems.

Main Findings

• Exact parity is often infeasible. The paper states that demographic parity cannot be achieved when base rates differ between groups: even a classifier that never makes errors (Ŷ = Y) will fail the parity ratio because P(Y=1|S=1)/P(Y=1|S=0) is not 1.

• Relaxing equality to a tolerance has two claimed benefits. The authors state that (i) by fine-tuning τ they can at least ensure a feasible solution exists, and (ii) because the binding equalities are relaxed, total accuracy (the weighted sum of false negatives and false positives) can be higher.

• The modified objective surfaces the ethical choice of error costs. In the formulation, α is the loss assigned to a false negative and β the loss assigned to a false positive, and W_i is the share of the population whose protected trait is v_i. The paper gives α/β >> 10 as a medical-style setting where missed cancer is far worse than a false alarm, and α/β << 0.1 as a credit-card setting where withholding credit does more social harm than granting it to someone who cannot repay.

• The relaxed formulation carries its own critique. The authors note that a designer aware of which constraints bind could set τ to 5 percent in a direction that further disadvantages one group, or set one group's false negative rejection rate 5 percent lower than others, which they describe as essentially affirmative action based on race and legally controversial.

• Group and individual fairness can be incompatible. The authors report that individual fairness (formulated with a (D,d)-Lipschitz constraint) implies group fairness if and only if the Wasserstein distance between the two groups' feature distributions is small. The paper states that if the two groups share very different feature distributions, the two notions cannot be achieved simultaneously. The paper does not report a numeric value for "small."

• Sacrificing cross-group individual fairness can buy group fairness. The paper describes a "fair affirmative action" algorithm that relaxes the Lipschitz condition so it need only hold within group S and within group T, achieving demographic parity up to a bias ε while preserving individual fairness within each group.

• Wasserstein distance predicts the fairness trade-off empirically. Citing work applying a disparate impact remover to the Adult dataset — which the paper describes as having a binary sensitive attribute (sex or race), five non-sensitive attributes (e.g., age and education), and a binary outcome label of whether income exceeds 50K a year — the authors report that a larger Wasserstein distance between attribute distributions leads to a larger decrease in individual fairness after the remover is applied. They also report the decrease is more likely when the large distance comes from a difference in mean rather than a difference in variance.

• Intersectional subgroup fairness is computationally and morally vague. The paper states that a survey could offer 6 options under religion, 7 under race, 20 under country of origin, and 3 under sexual orientation, which can introduce over one thousand categories. This raises three problems: no sufficient moral justification for ignoring intersections, the equivalent of adding a thousand or more linear constraints to a convex optimization problem (which could make the solution suboptimal or infeasible), and the large per-intersection sample size needed.

• Individual fairness alone is insufficient. The paper notes that a model classifying the same outcome for every individual would satisfy individual fairness but is clearly unfair, and cites a proposed index that decomposes overall fairness into between-group and within-group components, where improving one may harm the other.

• No empirical evaluation of the proposed formulation is reported. The paper presents the τ-bounded objective as a proposal and does not report accuracy or fairness measurements for it on any dataset or risk assessment tool.

Methodology in Plain English

The authors do not run a new experiment. They work in three modes. First, they survey and restate the standard definitions of fairness that other researchers have proposed, writing each one as a probability equation so the reader can see exactly what is being equalized. Second, they take the canonical group-fairness setup — train a classifier by minimizing the weighted sum of false negatives and false positives, subject to false negative rates being identical across all protected groups — and change the constraint so that group false negative rates only must fall within τ of each other. They then reason informally about why this relaxation should make the optimization easier to satisfy and why it should permit lower overall error, and they consider how a designer could abuse the τ knob. Third, they connect the technical discussion to policy by proposing three principles for deployment and supporting each with legal and process arguments, including a comparison to how US anti-discrimination law treats narrowly tailored remedies.

Why This Matters

Impact on research. The paper argues that the proliferation of incompatible fairness definitions means technical work cannot settle fairness questions on its own; choosing among definitions is a value-based decision. It reframes the accuracy-fairness trade-off as a question of how much slack in false negative rate disparities a designer is willing to accept, which is a parameter that can be discussed publicly rather than buried inside a constraint.

Real-world applications:

• Criminal justice risk assessment. The authors use COMPAS (Correctional Offender Management Profiling for Alternative Sanctions), which scores defendants' risk of recidivism using criminal history, education, income level, and family situation, as a running example of how same-threshold versus different-threshold decisions trade individual fairness against group fairness.
• Predictive policing. The paper cites work using a UK Crime API dataset showing that predictive policing algorithms can create biased feedback loops even with completely randomized synthesized historical data, because more police in previously affected areas make crime more identifiable there (the "Near Repeat Theory" logic).
• Consumer credit. The credit-card setting illustrates a loss ratio where α/β << 0.1, because withholding credit from deserving applicants harms household finances more than granting it to someone who cannot repay.
• Medical screening. The medical setting illustrates α/β >> 10, because a missed cancer diagnosis delays therapy while a false positive can be resolved by further testing.
• College admissions. The paper argues each institution needs a model based on its own history, contrasting a school in the South that historically discriminated against black students until the 1970s with a historically black college.

Industry relevance. Any organization deploying a scored decision — lending, hiring, insurance, benefits eligibility — faces the same choice of which fairness metric to satisfy and what to disclose about it. The paper's framework tells practitioners to state the unfairness definition precisely in both mathematical and plain language, to justify adjustments with historical evidence about the specific problem, and to avoid cookie-cutter remedies, which the authors warn risk legal challenges under US anti-discrimination law.

Future Directions

• Mathematically rigorous treatment of the trade-offs. The authors call for more rigorous analysis of the trade-offs between different types of unfairness than the informal reasoning offered here.
• Integrating legitimacy into the model. The paper suggests more work is needed to fold the political science concept of "legitimacy" into the fairness framework.
• Worked applications of the Three Pillars. The authors propose future papers analyzing real-world problems with the Three Pillars Model and comparing it against other frameworks.
• Deciding where to stop subdividing. The subgroup critique raises the open question of at what point one should stop considering further intersections of protected features, which the paper explicitly flags as unanswered.
• Choosing τ in practice. The paper does not report a method for selecting the tolerance bound or for validating a chosen value empirically; the 5 percent example is used only to illustrate the ethical stakes.

Target Audience

Researchers and graduate students in machine learning fairness who need a compact map of group, individual, and process fairness and where they conflict; policy analysts, judges, and legislative staff working on the legality of non-race-blind algorithmic adjustments; and practitioners at agencies and companies that deploy risk assessment or scored decision tools and need a deployment framework that connects metric choice to public legitimacy. Criminologists and political scientists studying the legitimacy of automated public decision systems will also find the three-pillar section directly relevant.

Authors’ abstract

Algorithmic fairness has grown rapidly as a research area, yet key concepts remain unsettled, especially in criminal justice. We review group, individual, and process fairness and map the conditions under which they conflict. We then develop a simple modification to standard group fairness. Rather than exact parity across protected groups, we minimize a weighted error loss while keeping differences in false negative rates within a small tolerance. This makes solutions easier to find, can raise predictive accuracy, and surfaces the ethical choice of error costs. We situate this proposal within three classes of critique: biased and incomplete data, latent affirmative action, and the explosion of subgroup constraints. Finally, we offer a practical framework for deployment in public decision systems built on three pillars: need-based decisions, Transparency and accountability, and narrowly tailored definitions and solutions. Together, these elements link technical design to legitimacy and provide actionable guidance for agencies that use risk assessment and related tools.