Google Research

Fair Performance Metric Elicitation

NeurIPS 2020

Abstract

What is a fair performance metric? We consider the choice of fairness metrics through the lens of metric elicitation -- a principled framework for selecting performance metrics that best reflect implicit preferences. The use of metric elicitation enables a practitioner to tune the performance and fairness metrics to the task, context, and population at hand. Specifically, we propose novel strategies to elicit fair performance metrics for multiclass classification problems with multiple sensitive groups that also includes selecting the tradeoff between performance and fairness. The proposed elicitation strategies require only relative preference and are robust to both finite sample and feedback noise.

Research Areas

Learn more about how we do research

We maintain a portfolio of research projects, providing individuals and teams the freedom to emphasize specific types of work