Introduction

Different fairness methods are available:

  • Unawareness

  • Demographic Parity

  • Equalized Odds

  • Predictive Rate Parity

  • Individual Fairness

  • Counterfactual fairness

Nomenclature (as used in [KLRS17] ):

  • \(A\) is a set of protected attributes of an individual

  • \(X\) other observable attributes of an individual

  • \(U\) relevant latent attributes which are not observed

  • \(Y\) predicted outcome

  • \(\hat{Y}\) the predictor, random variable that depends on \(A\) , \(X\) and \(U\)

Fairness Through Unawareness (FTU)

Definition (as in [KLRS17] )

An algorithm is fair so long as any protected attributes A are not explicitly used in the decision-making process. Any

Demographic parity (DP)

According to [KLRS17] :

A predictor \(\hat{Y}\) satisfies demographic parity if \(P(\hat{Y}|A=0)=P(\hat{Y}|A=1)\) .

This is explained in more detail in Demographic parity .

from typing import List

import pandas as pd

df_train = pd.read_csv("data/titanic/train.csv")
df_test = pd.read_csv("data/titanic/test.csv")
df_train.head()
PassengerId Survived Pclass Name Sex Age SibSp Parch Ticket Fare Cabin Embarked
0 1 0 3 Braund, Mr. Owen Harris male 22.0 1 0 A/5 21171 7.2500 NaN S
1 2 1 1 Cumings, Mrs. John Bradley (Florence Briggs Th... female 38.0 1 0 PC 17599 71.2833 C85 C
2 3 1 3 Heikkinen, Miss. Laina female 26.0 0 0 STON/O2. 3101282 7.9250 NaN S
3 4 1 1 Futrelle, Mrs. Jacques Heath (Lily May Peel) female 35.0 1 0 113803 53.1000 C123 S
4 5 0 3 Allen, Mr. William Henry male 35.0 0 0 373450 8.0500 NaN S
df_train = df_train[["Survived", "Pclass", "Sex", "Age", "Fare"]]
from sklearn import preprocessing

le = preprocessing.LabelEncoder()

df_train["Sex"] = le.fit_transform(df_train["Sex"])
df_train["Sex"] = df_train["Sex"].astype("category")
df_train["Survived"] = df_train["Survived"].astype("category")
df_train = df_train.dropna()
df_train.describe()
Pclass Age Fare
count 714.000000 714.000000 714.000000
mean 2.236695 29.699118 34.694514
std 0.838250 14.526497 52.918930
min 1.000000 0.420000 0.000000
25% 1.000000 20.125000 8.050000
50% 2.000000 28.000000 15.741700
75% 3.000000 38.000000 33.375000
max 3.000000 80.000000 512.329200
df_test["Sex"] = le.fit_transform(df_test["Sex"])
df_test["Sex"] = df_test["Sex"].astype("category")
df_test = df_test.dropna()

Train the model.

from sklearn.linear_model import LogisticRegression

model = LogisticRegression(random_state=23)
X = df_train.drop("Survived", axis=1)
y = df_train["Survived"]
model.fit(X, y)
LogisticRegression(random_state=23)
from fairlearn.metrics import demographic_parity_difference, equalized_odds_difference


def binary_accuracy(labels, scores, threshold=0.5):
    return ((scores >= threshold) == labels).mean()
bl_test_probs = model.predict_proba(X)[:, 1]
bl_test_labels = (bl_test_probs > 0.5).astype(float)
test_sex = df_train.Sex.values
test_survived = df_train.Survived.values
mask = test_sex == 1
model_ftu = LogisticRegression(random_state=23)
X_ftu = df_train.drop("Survived", axis=1).drop("Sex", axis=1)
y_ftu = df_train["Survived"]
model_ftu.fit(X_ftu, y_ftu)
LogisticRegression(random_state=23)
test_probs = model_ftu.predict_proba(X_ftu)[:, 1]
test_pred_labels = (test_probs > 0.5).astype(float)
# baseline metrics
bl_test_acc = binary_accuracy(test_survived, bl_test_labels)
bl_test_dpd = demographic_parity_difference(
    test_survived,
    bl_test_labels,
    sensitive_features=test_sex,
)
from plotutils import *

base_data = pd.DataFrame()
base_data["y"] = bl_test_probs
base_data["Sex"] = X.Sex
base_data.boxplot(by="Sex")

ftu_data = pd.DataFrame()
base_data["y"] = test_probs
base_data["Sex"] = X.Sex
base_data.boxplot(by="Sex")
<AxesSubplot:title={'center':'y'}, xlabel='[Sex]'>
_images/fairness-introduction_18_1.png _images/fairness-introduction_18_2.png

KLRS17 ( 1 , 2 , 3 )

Matt Kusner, Joshua Loftus, Chris Russell, and Ricardo Silva. Counterfactual fairness. Advances in Neural Information Processing Systems , 2017-Decem(Nips):4067–4077, 2017. arXiv:1703.06856 .