Daphne: A Flexible and Hybrid Scheduling Framework in Multi-Tenant Clusters

Xia, Y; Ren, R; Cai, H; Vasilakos, AV; Lv, Z

Daphne: A Flexible and Hybrid Scheduling Framework in Multi-Tenant Clusters

Xia, Y Ren, R Cai, H Vasilakos, AV Lv, Z

Permalink

Publisher:: IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
Publication Type:: Journal Article
Citation:: IEEE Transactions on Network and Service Management, 2018, 15, (1), pp. 330-343
Issue Date:: 2018-03-01

Closed Access

	Filename	Description	Size
	Daphne_A_Flexible_and_Hybrid_Scheduling_Framework_in_Multi-Tenant_Clusters.pdf	Published version	2.17 MB	Adobe PDF	View/Open

Copyright Clearance Process

Recently Added
In Progress
Closed Access

This item is closed access and not available.

Full metadata record

Field	Value	Language
dc.contributor.author	Xia, Y
dc.contributor.author	Ren, R
dc.contributor.author	Cai, H
dc.contributor.author	Vasilakos, AV
dc.contributor.author	Lv, Z
dc.date.accessioned	2022-08-29T22:35:37Z
dc.date.available	2022-08-29T22:35:37Z
dc.date.issued	2018-03-01
dc.identifier.citation	IEEE Transactions on Network and Service Management, 2018, 15, (1), pp. 330-343
dc.identifier.issn	1932-4537
dc.identifier.uri	http://hdl.handle.net/10453/161062
dc.description.abstract	Distributed computing technologies, as popularized by Hadoop, have been proliferating in Cloud and enterprise computing over ten years, with the capability of processing data across thousands of machines. There is a wide diversity of workloads in such large scale clusters shared by multi-Tenant. Hence, resource utilization and task scheduling become vital to performance and bring challenges to architecture designers. We present Daphne, a hybrid scheduling framework that strikes a tradeoff among three universal scheduling frameworks: 1) centralized scheduling; 2) loose coordination scheduling; and 3) fully distributed scheduling. Daphne defines a matching tree that forwards the application to the fittest scheduler according to the characteristic and priority of the application. Besides, Daphne utilizes resource prediction to increase task throughput significantly while it does not interfere with any running workload. We implement Daphne based on YARN and demonstrate that Daphne improves task throughput by nearly 17%.
dc.language	English
dc.publisher	IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
dc.relation.ispartof	IEEE Transactions on Network and Service Management
dc.relation.isbasedon	10.1109/TNSM.2017.2777885
dc.rights	info:eu-repo/semantics/closedAccess
dc.subject	0805 Distributed Computing, 0906 Electrical and Electronic Engineering, 1005 Communications Technologies
dc.subject.classification	Networking & Telecommunications
dc.title	Daphne: A Flexible and Hybrid Scheduling Framework in Multi-Tenant Clusters
dc.type	Journal Article
utslib.citation.volume	15
utslib.for	0805 Distributed Computing
utslib.for	0906 Electrical and Electronic Engineering
utslib.for	1005 Communications Technologies
pubs.organisational-group	/University of Technology Sydney
pubs.organisational-group	/University of Technology Sydney/Faculty of Engineering and Information Technology
pubs.organisational-group	/University of Technology Sydney/Faculty of Engineering and Information Technology/School of Electrical and Data Engineering
utslib.copyright.status	closed_access	*
dc.date.updated	2022-08-29T22:35:36Z
pubs.issue	1
pubs.publication-status	Published
pubs.volume	15
utslib.citation.issue	1

Abstract:

Distributed computing technologies, as popularized by Hadoop, have been proliferating in Cloud and enterprise computing over ten years, with the capability of processing data across thousands of machines. There is a wide diversity of workloads in such large scale clusters shared by multi-Tenant. Hence, resource utilization and task scheduling become vital to performance and bring challenges to architecture designers. We present Daphne, a hybrid scheduling framework that strikes a tradeoff among three universal scheduling frameworks: 1) centralized scheduling; 2) loose coordination scheduling; and 3) fully distributed scheduling. Daphne defines a matching tree that forwards the application to the fittest scheduler according to the characteristic and priority of the application. Besides, Daphne utilizes resource prediction to increase task throughput significantly while it does not interfere with any running workload. We implement Daphne based on YARN and demonstrate that Daphne improves task throughput by nearly 17%.

Please use this identifier to cite or link to this item:

http://hdl.handle.net/10453/161062