Optimizing the Numbers of Queries and Replies in Convex Federated Learning with Differential Privacy

Zhou, Y; Liu, X; Fu, Y; Wu, D; Wang, JH; Yu, S

Optimizing the Numbers of Queries and Replies in Convex Federated Learning with Differential Privacy

Zhou, Y Liu, X Fu, Y Wu, D Wang, JH Yu, S

Permalink

Publisher:: Institute of Electrical and Electronics Engineers (IEEE)
Publication Type:: Journal Article
Citation:: IEEE Transactions on Dependable and Secure Computing, 2023, 20, (6), pp. 4823-4837
Issue Date:: 2023-11-01

Closed Access

	Filename	Description	Size
	Optimizing_the_Numbers_of_Queries_and_Replies_in_Convex_Federated_Learning_With_Differential_Privacy.pdf	Published version	1.82 MB	Adobe PDF	View/Open

Copyright Clearance Process

Recently Added
In Progress
Closed Access

This item is closed access and not available.

Full metadata record

Field	Value	Language
dc.contributor.author	Zhou, Y
dc.contributor.author	Liu, X
dc.contributor.author	Fu, Y
dc.contributor.author	Wu, D
dc.contributor.author	Wang, JH
dc.contributor.author	Yu, S https://orcid.org/0000-0003-4485-6743
dc.date.accessioned	2024-03-12T04:20:46Z
dc.date.available	2024-03-12T04:20:46Z
dc.date.issued	2023-11-01
dc.identifier.citation	IEEE Transactions on Dependable and Secure Computing, 2023, 20, (6), pp. 4823-4837
dc.identifier.issn	1545-5971
dc.identifier.issn	1941-0018
dc.identifier.uri	http://hdl.handle.net/10453/176565
dc.description.abstract	Federated learning (FL) empowers distributed clients to collaboratively train a shared machine learning model through exchanging parameter information. Despite the fact that FL can protect clients' raw data, malicious users can still crack original data with disclosed parameters. To amend this flaw, differential privacy (DP) is incorporated into FL clients to disturb original parameters, which however can significantly impair the accuracy of the trained model. In this work, we study an imperative question which has been vastly overlooked by existing works: what are the optimal numbers of queries and replies in FL with DP so that the final model accuracy is maximized. In FL, the parameter server (PS) needs to query participating clients for multiple global iterations to complete training. Each client responds a query from the PS by conducting a local iteration. We consider FL that will uniformly and randomly select participating clients to conduct local iterations with the FedSGD algorithm. Our work investigates how many times the PS should query clients and how many times each client should reply the PS by incorporating two most extensively used DP mechanisms (i.e., the Laplace mechanism and Gaussian mechanisms). Through conducting convergence rate analysis, we can determine the optimal numbers of queries and replies in FL with DP so that the final model accuracy can be maximized. Finally, extensive experiments are conducted with publicly available datasets: MNIST and FEMNIST, to verify our analysis and the results demonstrate that properly setting the numbers of queries and replies can significantly improve the final model accuracy in FL with DP.
dc.language	en
dc.publisher	Institute of Electrical and Electronics Engineers (IEEE)
dc.relation.ispartof	IEEE Transactions on Dependable and Secure Computing
dc.relation.isbasedon	10.1109/TDSC.2023.3234599
dc.rights	info:eu-repo/semantics/closedAccess
dc.subject	0803 Computer Software, 0804 Data Format, 0805 Distributed Computing
dc.subject.classification	Strategic, Defence & Security Studies
dc.subject.classification	4604 Cybersecurity and privacy
dc.subject.classification	4606 Distributed computing and systems software
dc.title	Optimizing the Numbers of Queries and Replies in Convex Federated Learning with Differential Privacy
dc.type	Journal Article
utslib.citation.volume	20
utslib.for	0803 Computer Software
utslib.for	0804 Data Format
utslib.for	0805 Distributed Computing
pubs.organisational-group	University of Technology Sydney
pubs.organisational-group	University of Technology Sydney/Faculty of Engineering and Information Technology
pubs.organisational-group	University of Technology Sydney/Faculty of Engineering and Information Technology/School of Computer Science
pubs.organisational-group	University of Technology Sydney/Strength - CCSP - Centre for Cyber Security and Privacy
utslib.copyright.status	closed_access	*
dc.date.updated	2024-03-12T04:20:45Z
pubs.issue	6
pubs.publication-status	Published
pubs.volume	20
utslib.citation.issue	6

Abstract:

Federated learning (FL) empowers distributed clients to collaboratively train a shared machine learning model through exchanging parameter information. Despite the fact that FL can protect clients' raw data, malicious users can still crack original data with disclosed parameters. To amend this flaw, differential privacy (DP) is incorporated into FL clients to disturb original parameters, which however can significantly impair the accuracy of the trained model. In this work, we study an imperative question which has been vastly overlooked by existing works: what are the optimal numbers of queries and replies in FL with DP so that the final model accuracy is maximized. In FL, the parameter server (PS) needs to query participating clients for multiple global iterations to complete training. Each client responds a query from the PS by conducting a local iteration. We consider FL that will uniformly and randomly select participating clients to conduct local iterations with the FedSGD algorithm. Our work investigates how many times the PS should query clients and how many times each client should reply the PS by incorporating two most extensively used DP mechanisms (i.e., the Laplace mechanism and Gaussian mechanisms). Through conducting convergence rate analysis, we can determine the optimal numbers of queries and replies in FL with DP so that the final model accuracy can be maximized. Finally, extensive experiments are conducted with publicly available datasets: MNIST and FEMNIST, to verify our analysis and the results demonstrate that properly setting the numbers of queries and replies can significantly improve the final model accuracy in FL with DP.

Please use this identifier to cite or link to this item:

http://hdl.handle.net/10453/176565