Omer Reingold (Stanford) - Algorithmic Fairness, Loss Minimization and Outcome Indistinguishability

Training a predictor to minimize a loss function fixed in advance is the dominant paradigm in machine learning. However, loss minimization by itself might fail to satisfy properties that come naturally in the context of algorithmic fairness. To remedy this, multi-group fairness notions such as multicalibration have been proposed, which require the predictor to share certain statistical properties of the ground truth, even when conditioned on a rich family of subgroups. These notions could be understood from the perspective of computational indistinguishability through the notion of outcome indistinguishability where a predictor can be viewed as giving a model of events that cannot be refused from empiric evidence within some computational bound. While differently motivated, this alternative paradigm for training predictors gives unexpected consequences, including: Practical methods for learning in a heterogeneous population, employed in the field to predict COVID-19 complications at a very early stage of the pandemic. A computational perspective on the meaning of individual probabilities. A rigorous new paradigm for loss minimization in machine learning, through the notion of omni predictors, that simultaneously applies to a wide class of loss-functions, allowing the specific loss function to be ignored at the time of learning. A method for adapting a statistical study on one probability distribution to another, which is blind to the target distribution at the time of inference and is competitive with wide-spread methods based on propensity scoring.

Based on a sequence of works joint with (subsets of) Cynthia Dwork, , Shafi Goldwasser, Parikshit Gopalan, Úrsula Hébert-Johnson, Adam Kalai, Christoph Kern, Michael P. Kim, Frauke Kreuter, Guy N. Rothblum, Vatsal Sharan, Udi Wieder, Gal Yona.