The Quest for Reliable Metrics of Responsible AI

AI summary

Overview

Research area: Responsible AI, specifically the evaluation of fairness in Recommender Systems (RSs) and the reliability of the metrics used to measure it. The paper is categorized on arXiv under AI Safety & Ethics (cs.CY) and is titled "The Quest for Reliable Metrics of Responsible AI."

Technical level: Beginner-Friendly. This is a reflection and guidelines paper rather than an experimental study. It condenses the authors' earlier theoretical and empirical work into a set of practical questions, and does not introduce new datasets, benchmarks, or model results.

Scope (one sentence): The paper distils the authors' prior findings on the flaws of fairness metrics for recommender systems into a short, non-exhaustive checklist for building reliable evaluation metrics of responsible AI, including AI in Science (AIS).

What This Paper Is About

Progress in responsible AI is usually measured with evaluation metrics, but far less attention has been paid to whether those metrics themselves are robust and reliable. The authors argue that a metric score can be misleading, unstable, or uninterpretable, and that this matters because metrics are the main scalable tool guiding responsible AI development, especially early on and in place of costly user studies. The paper's goal is to generalize lessons learned from auditing recommender system fairness metrics into a set of guiding questions for anyone formulating new responsible AI metrics.

Key Contributions

1. A synthesis of prior metric audits. The paper consolidates insights from four previous works by the authors (Rampisela et al., 2024a, 2024b, 2025a, 2025b) on the robustness of recommender system fairness metrics, plus a joint effectiveness-fairness metric (Rampisela et al., 2025c).
2. A set of five guiding questions for developing reliable metrics of responsible AI, covering excluded input cases, metric range and interpretation, what inputs produce minimum and maximum scores, sensitivity to input changes, and redundancy with existing metrics.
3. A reframing of fairness evaluation practice into practical guidance for metric selection, including using corrected metrics, interpreting scores cautiously, avoiding redundant measures, and evaluating both group and individual fairness.
4. A call to include evaluation metrics in AI policy discussions, noting that quantification is an important part of regulation and policy-making yet metrics are frequently absent from current AI policy debate.

Main Findings

• Some fairness metrics are mathematically flawed. The authors report that certain metrics crash during computation due to invalid mathematical operations such as division by zero, producing no usable score at all.
• Score ranges are often unknown or unreachable. For many metrics the range is unknown, or the theoretical maximum and minimum are not achievable. The paper gives the illustration of a metric nominally ranging over [0, 1] where 0 is fairest and 1 is unfairest, but which in practice can only produce values between 0.3 and 0.6 — so a score of 0.5 reads as fair under [0, 1] but unfair under [0.3, 0.6].
• It is sometimes unclear what input yields the extreme scores. In some cases it is not even known what kind of input produces the minimum or maximum metric value, so the fairest and unfairest possible scenarios according to the metric are poorly understood.
• Scores can be compressed and insensitive. Empirically, some metrics score very low (close to 0) regardless of the actual fairness level. This compressed range and limited sensitivity create the illusion of an extremely fair input even when the input is not fair.
• Some metrics are redundant. Certain metrics yield conclusions similar to other existing metrics, suggesting that computing one of the similar measures may suffice.
• Granularity levels are not interchangeable. The authors show that group fairness metrics cannot be used as a proxy for individual fairness, so both granularities must be considered for a comprehensive view of fairness.
• Corrections and new metrics exist. The authors corrected existing metrics by redefining formulations to avoid computation crashes and by applying min-max normalisation so that 0 maps to the fairest possible case and 1 to the unfairest, and they proposed a new metric that jointly evaluates recommender system effectiveness and fairness because existing metrics cannot quantify both simultaneously. Source code for both is released publicly.
• Scope of the guidance. The guidelines are explicitly described as non-exhaustive and as the bare minimum for reliable metrics, and are stated to apply to a broad spectrum of AI applications including AI in Science.

Methodology in Plain English

This is not a study that runs new experiments. Instead, the authors look back at their own body of work, in which they examined recommender system fairness metrics both theoretically and empirically, and they generalize the problems they found into reusable questions.

The underlying approach in that prior work was to interrogate each metric directly: check whether its formula can be computed at all, work out what range of values it can actually produce, determine which inputs produce its best and worst scores, test how much the score moves when the input changes, and compare it against other metrics to see whether it says anything new. The authors also corrected broken metrics and proposed a metric that measures effectiveness and fairness together. The result of this reflection is the paper's five-question checklist plus practical advice on choosing metrics.

Why This Matters

Impact on research. Metrics are the primary scalable mechanism for judging whether an AI system respects responsible AI principles, and they are especially important in early development stages when user studies are too expensive. If the metrics themselves are flawed, compressed, or uninterpretable, the research conclusions drawn from them can be wrong. The paper pushes the field to audit its instruments, not just its systems. It also notes that the issue is not confined to recommender systems, since AI in Science will inevitably require new metrics for responsible AI aspects.

Real-world applications:

• Job recommendation. An unfair recommender might show historically marginalised groups, such as women, only lower-paying jobs while reserving highly paid positions for the historically dominant group, contributing to the exacerbation of gender pay gaps.
• Scientific paper and citation recommendation (AIS). A system might perform very well for one discipline, such as computer science, but poorly for others, such as Nordic studies, due to training data imbalance between fields of study, hindering scientific development.
• Geographic exposure imbalance in science. A system that overpromotes articles by researchers from economically developed countries may give less exposure to researchers from other countries, producing a less inclusive understanding of science, particularly in social sciences and humanities where cultural context matters.
• Regulatory and policy contexts. The paper frames reliable quantification as an essential input to AI regulation and policy-making, especially in high-stakes settings such as AI in Science.

Industry relevance. Practitioners selecting fairness metrics from the more than 30 that have been used for recommender systems currently have little guidance on which are trustworthy. The paper's checklist and its recommendation to avoid redundant measures, interpret scores cautiously, and cover both group and individual fairness give teams a concrete basis for auditing the metrics they report.

Future Directions

1. Collaboration across stakeholder groups. The authors propose working with AI users, technical actors, social scientists, ethics experts, policy makers, and governmental agencies to determine which responsible AI evaluation metrics should be incorporated into AI policies and regulations.
2. Determining which responsible AI aspects are critical for AIS specifically. The paper identifies this as an open question for a future collaborative effort rather than something resolved here.
3. Auditing existing evaluation metrics. A proposed next step is to audit current metrics to verify their reliability in measuring the responsible AI aspects they claim to measure.
4. Producing more concrete, measurable guidelines. The authors expect such collaboration to yield guidelines for responsible AI development that are more concrete and measurable than the five questions offered here.

Target Audience

Researchers and practitioners who design or use evaluation metrics for responsible AI; recommender system researchers working on fairness and bias; AI in Science developers; and policy makers, regulators, and ethics experts who rely on quantified evaluation in AI governance. It is most useful for readers who want a short, low-jargon checklist rather than technical derivations, since the detailed theoretical and empirical analyses live in the authors' cited prior papers.

Authors’ abstract

The development of Artificial Intelligence (AI), including AI in Science (AIS), should be done following the principles of responsible AI. Progress in responsible AI is often quantified through evaluation metrics, yet there has been less work on assessing the robustness and reliability of the metrics themselves. We reflect on prior work that examines the robustness of fairness metrics for recommender systems as a type of AI application and summarise their key takeaways into a set of non-exhaustive guidelines for developing reliable metrics of responsible AI. Our guidelines apply to a broad spectrum of AI applications, including AIS.