Differentially Private Regression and Classification with Sparse Gaussian Processes

Michael Thomas Smith; Mauricio A. Álvarez; Neil D. Lawrence

Back to publications

Differentially Private Regression and Classification with Sparse Gaussian Processes

Michael Thomas Smith, Mauricio A. Álvarez, Neil D. Lawrence

Journal of Machine Learning Research, 22(188):1-41, 2021.

Abstract

A continuing challenge for machine learning is providing methods to perform computation on data while ensuring the data remains private. In this paper we build on the provable privacy guarantees of differential privacy which has been combined with Gaussian processes through the previously published cloaking method, an approach that tackles the problem of providing privacy for the outputs of a training set. In this paper we solve several shortcomings of this method, starting with the problem of predictions in regions with low data density. We experiment with the use of inducing points to provide a sparse approximation and show that these can provide robust differential privacy in outlier areas and at higher dimensions. We then look at classification, and modify the Laplace approximation approach to provide differentially private predictions. We then combine this with the sparse approximation and demonstrate the capability to perform classification in high dimensions. We finally explore the issue of hyperparameter selection and develop a method for their private selection. This paper and associated libraries provide a robust toolkit for combining differential privacy and Gaussian processes in a practical manner.

Links

Cite this Paper

BibTeX


@Article{Smith-differentially21,
  title = 	 {Differentially Private Regression and Classification with Sparse Gaussian Processes},
  author = 	 {Smith, Michael Thomas and Álvarez, Mauricio A. and Lawrence, Neil D.},
  journal =      {Journal of Machine Learning Research},
  pages = 	 {1--41},
  year = 	 {2021},
  volume = 	 {22},
  number =       {188},
  pdf = 	 {https://jmlr.org/papers/volume22/19-017/19-017.pdf},
  url = 	 {/publications/differentially-private-regression-and-classificaiton-with-sparse-gaussian-processes.html},
  abstract = 	 {A continuing challenge for machine learning is providing methods to perform computation on data while ensuring the data remains private. In this paper we build on the provable privacy guarantees of differential privacy which has been combined with Gaussian processes through the previously published cloaking method, an approach that tackles the problem of providing privacy for the outputs of a training set. In this paper we solve several shortcomings of this method, starting with the problem of predictions in regions with low data density. We experiment with the use of inducing points to provide a sparse approximation and show that these can provide robust differential privacy in outlier areas and at higher dimensions. We then look at classification, and modify the Laplace approximation approach to provide differentially private predictions. We then combine this with the sparse approximation and demonstrate the capability to perform classification in high dimensions. We finally explore the issue of hyperparameter selection and develop a method for their private selection. This paper and associated libraries provide a robust toolkit for combining differential privacy and Gaussian processes in a practical manner.
}
}

Endnote

%0 Journal Article
%T Differentially Private Regression and Classification with Sparse Gaussian Processes
%A Michael Thomas Smith
%A Mauricio A. Álvarez
%A Neil D. Lawrence
%J Journal of Machine Learning Research
%D 2021	
%F Smith-differentially21
%P 1--41
%U /publications/differentially-private-regression-and-classificaiton-with-sparse-gaussian-processes.html
%V 22
%N 188
%X A continuing challenge for machine learning is providing methods to perform computation on data while ensuring the data remains private. In this paper we build on the provable privacy guarantees of differential privacy which has been combined with Gaussian processes through the previously published cloaking method, an approach that tackles the problem of providing privacy for the outputs of a training set. In this paper we solve several shortcomings of this method, starting with the problem of predictions in regions with low data density. We experiment with the use of inducing points to provide a sparse approximation and show that these can provide robust differential privacy in outlier areas and at higher dimensions. We then look at classification, and modify the Laplace approximation approach to provide differentially private predictions. We then combine this with the sparse approximation and demonstrate the capability to perform classification in high dimensions. We finally explore the issue of hyperparameter selection and develop a method for their private selection. This paper and associated libraries provide a robust toolkit for combining differential privacy and Gaussian processes in a practical manner.

RIS


TY  - JOUR
TI  - Differentially Private Regression and Classification with Sparse Gaussian Processes
AU  - Michael Thomas Smith
AU  - Mauricio A. Álvarez
AU  - Neil D. Lawrence
DA  - 2021/10/08	
ID  - Smith-differentially21
VL  - 22
IS  - 188
SP  - 1
EP  - 41
L1  - https://jmlr.org/papers/volume22/19-017/19-017.pdf
UR  - /publications/differentially-private-regression-and-classificaiton-with-sparse-gaussian-processes.html
AB  - A continuing challenge for machine learning is providing methods to perform computation on data while ensuring the data remains private. In this paper we build on the provable privacy guarantees of differential privacy which has been combined with Gaussian processes through the previously published cloaking method, an approach that tackles the problem of providing privacy for the outputs of a training set. In this paper we solve several shortcomings of this method, starting with the problem of predictions in regions with low data density. We experiment with the use of inducing points to provide a sparse approximation and show that these can provide robust differential privacy in outlier areas and at higher dimensions. We then look at classification, and modify the Laplace approximation approach to provide differentially private predictions. We then combine this with the sparse approximation and demonstrate the capability to perform classification in high dimensions. We finally explore the issue of hyperparameter selection and develop a method for their private selection. This paper and associated libraries provide a robust toolkit for combining differential privacy and Gaussian processes in a practical manner.

ER  -

APA


Smith, M.T., Álvarez, M.A. & Lawrence, N.D.. (2021). Differentially Private Regression and Classification with Sparse Gaussian Processes. Journal of Machine Learning Research 22(188):1-41 Available from /publications/differentially-private-regression-and-classificaiton-with-sparse-gaussian-processes.html.