A Pretreatment Workflow Scheduling Approach for Big Data Applications in Multicloud Environments

Lin, B; Guo, W; Xiong, N; Chen, G; Vasilakos, AV; Zhang, H

A Pretreatment Workflow Scheduling Approach for Big Data Applications in Multicloud Environments

Lin, B Guo, W Xiong, N Chen, G Vasilakos, AV Zhang, H

Permalink

Publisher:: IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
Publication Type:: Journal Article
Citation:: IEEE Transactions on Network and Service Management, 2016, 13, (3), pp. 581-594
Issue Date:: 2016-09-01

Closed Access

	Filename	Description	Size
	A_Pretreatment_Workflow_Scheduling_Approach_for_Big_Data_Applications_in_Multicloud_Environments.pdf	Published version	1.72 MB	Adobe PDF	View/Open

Copyright Clearance Process

Recently Added
In Progress
Closed Access

This item is closed access and not available.

Full metadata record

Field	Value	Language
dc.contributor.author	Lin, B
dc.contributor.author	Guo, W
dc.contributor.author	Xiong, N
dc.contributor.author	Chen, G
dc.contributor.author	Vasilakos, AV
dc.contributor.author	Zhang, H
dc.date.accessioned	2022-08-10T03:48:38Z
dc.date.available	2022-08-10T03:48:38Z
dc.date.issued	2016-09-01
dc.identifier.citation	IEEE Transactions on Network and Service Management, 2016, 13, (3), pp. 581-594
dc.identifier.issn	1932-4537
dc.identifier.uri	http://hdl.handle.net/10453/159841
dc.description.abstract	The rapid development of the latest distributed computing paradigm, i.e., cloud computing, generates a highly fragmented cloud market composed of numerous cloud providers and offers tremendous parallel computing ability to handle big data problems. One of the biggest challenges in multiclouds is efficient workflow scheduling. Although the workflow scheduling problem has been studied extensively, there are still very few primal works tailored for multicloud environments. Moreover, the existing research works either fail to satisfy the quality of service (QoS) requirements, or do not consider some fundamental features of cloud computing such as heterogeneity and elasticity of computing resources. In this paper, a scheduling algorithm, which is called multiclouds partial critical paths with pretreatment (MCPCPP), for big data workflows in multiclouds is presented. This algorithm incorporates the concept of partial critical paths, and aims to minimize the execution cost of workflow while satisfying the defined deadline constraint. Our approach takes into consideration the essential characteristics of multiclouds such as the charge per time interval, various instance types from different cloud providers, as well as homogeneous intrabandwidth vs. heterogeneous interbandwidth. Various types of workflows are used for evaluation purpose and our experimental results show that the MCPCPP is promising.
dc.language	English
dc.publisher	IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
dc.relation.ispartof	IEEE Transactions on Network and Service Management
dc.relation.isbasedon	10.1109/TNSM.2016.2554143
dc.rights	info:eu-repo/semantics/closedAccess
dc.subject	0805 Distributed Computing, 0906 Electrical and Electronic Engineering, 1005 Communications Technologies
dc.subject.classification	Networking & Telecommunications
dc.title	A Pretreatment Workflow Scheduling Approach for Big Data Applications in Multicloud Environments
dc.type	Journal Article
utslib.citation.volume	13
utslib.for	0805 Distributed Computing
utslib.for	0906 Electrical and Electronic Engineering
utslib.for	1005 Communications Technologies
pubs.organisational-group	/University of Technology Sydney
pubs.organisational-group	/University of Technology Sydney/Faculty of Engineering and Information Technology
pubs.organisational-group	/University of Technology Sydney/Faculty of Engineering and Information Technology/School of Electrical and Data Engineering
utslib.copyright.status	closed_access	*
dc.date.updated	2022-08-10T03:48:37Z
pubs.issue	3
pubs.publication-status	Published
pubs.volume	13
utslib.citation.issue	3

Abstract:

The rapid development of the latest distributed computing paradigm, i.e., cloud computing, generates a highly fragmented cloud market composed of numerous cloud providers and offers tremendous parallel computing ability to handle big data problems. One of the biggest challenges in multiclouds is efficient workflow scheduling. Although the workflow scheduling problem has been studied extensively, there are still very few primal works tailored for multicloud environments. Moreover, the existing research works either fail to satisfy the quality of service (QoS) requirements, or do not consider some fundamental features of cloud computing such as heterogeneity and elasticity of computing resources. In this paper, a scheduling algorithm, which is called multiclouds partial critical paths with pretreatment (MCPCPP), for big data workflows in multiclouds is presented. This algorithm incorporates the concept of partial critical paths, and aims to minimize the execution cost of workflow while satisfying the defined deadline constraint. Our approach takes into consideration the essential characteristics of multiclouds such as the charge per time interval, various instance types from different cloud providers, as well as homogeneous intrabandwidth vs. heterogeneous interbandwidth. Various types of workflows are used for evaluation purpose and our experimental results show that the MCPCPP is promising.

Please use this identifier to cite or link to this item:

http://hdl.handle.net/10453/159841