Differentially private model personalization
Files
Date
2021-12-06
DOI
Authors
Smith, Adam
Jain, Prateek
Rush, Keith
Song, Shuang
Thakurta, Abhradeep G.
Version
OA Version
Citation
A. Smith, P. Jain, J. Rush, S. Song, A.G. Thakurta. 2021. "Differentially Private Model Personalization." Advances in Neural Information Processing Systems 34 (NeurIPS 2021)
Abstract
We study personalization of supervised learning with user-level differential privacy. Consider a setting with many users, each of whom has a training data set drawn from their own distribution Pi . Assuming some shared structure among the problems Pi, can users collectively learn the shared structure---and solve their tasks better than they could individually---while preserving the privacy of their data? We formulate this question using joint, user-level differential privacy---that is, we control what is leaked about each user's entire data set. We provide algorithms that exploit popular non-private approaches in this domain like the Almost-No-Inner-Loop (ANIL) method, and give strong user-level privacy guarantees for our general approach. When the problems Pi are linear regression problems with each user's regression vector lying in a common, unknown low-dimensional subspace, we show that our efficient algorithms satisfy nearly optimal estimation error guarantees. We also establish a general, information-theoretic upper bound via an exponential mechanism-based algorithm.