Skip to main content

Learning Privately over Distributed Features: An ADMM Sharing Approach

Hu, Yaochen and Liu, Peng and Kong, Linglong and Niu, Di (2019) Learning Privately over Distributed Features: An ADMM Sharing Approach. Working paper. Submitted to The Thirty-Fourth AAAI Conference on Artificial Intelligence (AAAI-20) (Submitted) (Access to this publication is currently restricted. You may be able to access a copy if URLs are provided)

PDF - Pre-print
Restricted to Repository staff only
Contact us about this Publication Download (348kB)
[img]

Abstract

Distributed machine learning has been widely studied in order to handle exploding amount of data. In this paper, we study an important yet less visited distributed learning problem where features are inherently distributed or vertically partitioned among multiple parties, and sharing of raw data or model parameters among parties is prohibited due to privacy concerns. We propose an ADMM sharing framework to approach risk minimization over distributed features, where each party only needs to share a single value for each sample in the training process, thus minimizing the data leakage risk. We establish convergence and iteration complexity results for the proposed parallel ADMM algorithm under nonconvex loss. We further introduce a novel differentially private ADMM sharing algorithm and bound the privacy guarantee with carefully designed noise perturbation. The experiments based on a prototype system shows that the proposed ADMM algorithms converge efficiently in a robust fashion, demonstrating advantage over gradient based methods especially for data set with high dimensional feature spaces.

Item Type: Monograph (Working paper)
Subjects: Q Science > Q Science (General) > Q335 Artificial intelligence
Divisions: Faculties > Sciences > School of Mathematics Statistics and Actuarial Science > Statistics
Depositing User: Peng Liu
Date Deposited: 06 Sep 2019 11:24 UTC
Last Modified: 09 Sep 2019 14:38 UTC
Resource URI: https://kar.kent.ac.uk/id/eprint/76239 (The current URI for this page, for reference purposes)
Liu, Peng: https://orcid.org/0000-0002-0492-0029
  • Depositors only (login required):

Downloads

Downloads per month over past year