Automated discovery of trade-off between utility, privacy and fairness in machine learning models

Bogdan Ficiu; Neil D. Lawrence; Andrei Paleyes

Back to publications

Automated discovery of trade-off between utility, privacy and fairness in machine learning models

Bogdan Ficiu, Neil D. Lawrence, Andrei Paleyes

3rd Workshop on Bias and Fairness in AI (BIAS), ECML 2023, 2023.

Abstract

Machine learning models are deployed as a central component in decision making and policy operations with direct impact on individuals’ lives. In order to act ethically and comply with government regulations, these models need to make fair decisions and protect the users’ privacy. However, such requirements can come with decrease in models’ performance compared to their potentially biased, privacy-leaking counterparts. Thus the trade-off between fairness, privacy and performance of ML models emerges, and practitioners need a way of quantifying this trade-off to enable deployment decisions. In this work we interpret this trade-off as a multi-objective optimization problem, and propose PFairDP, a pipeline that uses Bayesian optimization for discovery of Pareto-optimal points between fairness, privacy and utility of ML models. We show how PFairDP can be used to replicate known results that were achieved through manual constraint setting process. We further demonstrate effectiveness of PFairDP with experiments on multiple models and datasets.

Links

Cite this Paper

BibTeX


@InProceedings{publications/automatic-trade-off-between-utility-privacy-and-fairness-in-machine-learning-models,
  title = 	 {Automated discovery of trade-off between utility, privacy and fairness in machine learning models},
  author = 	 {Ficiu, Bogdan and Lawrence, Neil D. and Paleyes, Andrei},
  booktitle = 	 {3rd Workshop on Bias and Fairness in AI (BIAS), ECML 2023},
  year = 	 {2023},
  url = 	 {/publications/automatic-trade-off-between-utility-privacy-and-fairness-in-machine-learning-models.html},
  abstract = 	 {Machine learning models are deployed as a central component in decision making and policy operations with direct impact on individuals' lives. In order to act ethically and comply with government regulations, these models need to make fair decisions and protect the users' privacy. However, such requirements can come with decrease in models' performance compared to their potentially biased, privacy-leaking counterparts. Thus the trade-off between fairness, privacy and performance of ML models emerges, and practitioners need a way of quantifying this trade-off to enable deployment decisions. In this work we interpret this trade-off as a multi-objective optimization problem, and propose PFairDP, a pipeline that uses Bayesian optimization for discovery of Pareto-optimal points between fairness, privacy and utility of ML models. We show how PFairDP can be used to replicate known results that were achieved through manual constraint setting process. We further demonstrate effectiveness of PFairDP with experiments on multiple models and datasets.
}
}

Endnote

%0 Conference Paper
%T Automated discovery of trade-off between utility, privacy and fairness in machine learning models
%A Bogdan Ficiu
%A Neil D. Lawrence
%A Andrei Paleyes
%B 3rd Workshop on Bias and Fairness in AI (BIAS), ECML 2023
%D 2023	
%F publications/automatic-trade-off-between-utility-privacy-and-fairness-in-machine-learning-models
%U /publications/automatic-trade-off-between-utility-privacy-and-fairness-in-machine-learning-models.html
%X Machine learning models are deployed as a central component in decision making and policy operations with direct impact on individuals' lives. In order to act ethically and comply with government regulations, these models need to make fair decisions and protect the users' privacy. However, such requirements can come with decrease in models' performance compared to their potentially biased, privacy-leaking counterparts. Thus the trade-off between fairness, privacy and performance of ML models emerges, and practitioners need a way of quantifying this trade-off to enable deployment decisions. In this work we interpret this trade-off as a multi-objective optimization problem, and propose PFairDP, a pipeline that uses Bayesian optimization for discovery of Pareto-optimal points between fairness, privacy and utility of ML models. We show how PFairDP can be used to replicate known results that were achieved through manual constraint setting process. We further demonstrate effectiveness of PFairDP with experiments on multiple models and datasets.

RIS


TY  - CPAPER
TI  - Automated discovery of trade-off between utility, privacy and fairness in machine learning models
AU  - Bogdan Ficiu
AU  - Neil D. Lawrence
AU  - Andrei Paleyes
BT  - 3rd Workshop on Bias and Fairness in AI (BIAS), ECML 2023
DA  - 2023/11/27	
ID  - publications/automatic-trade-off-between-utility-privacy-and-fairness-in-machine-learning-models
UR  - /publications/automatic-trade-off-between-utility-privacy-and-fairness-in-machine-learning-models.html
AB  - Machine learning models are deployed as a central component in decision making and policy operations with direct impact on individuals' lives. In order to act ethically and comply with government regulations, these models need to make fair decisions and protect the users' privacy. However, such requirements can come with decrease in models' performance compared to their potentially biased, privacy-leaking counterparts. Thus the trade-off between fairness, privacy and performance of ML models emerges, and practitioners need a way of quantifying this trade-off to enable deployment decisions. In this work we interpret this trade-off as a multi-objective optimization problem, and propose PFairDP, a pipeline that uses Bayesian optimization for discovery of Pareto-optimal points between fairness, privacy and utility of ML models. We show how PFairDP can be used to replicate known results that were achieved through manual constraint setting process. We further demonstrate effectiveness of PFairDP with experiments on multiple models and datasets.

ER  -

APA


Ficiu, B., Lawrence, N.D. & Paleyes, A.. (2023). Automated discovery of trade-off between utility, privacy and fairness in machine learning models. 3rd Workshop on Bias and Fairness in AI (BIAS), ECML 2023 Available from /publications/automatic-trade-off-between-utility-privacy-and-fairness-in-machine-learning-models.html.