Logical query optimization for Cloudera Impala system

Ma, J; Chen, L; Lv, M; Yang, Y; Zhao, Y; Wu, Y; Wang, J

Logical query optimization for Cloudera Impala system

Ma, J Chen, L Lv, M Yang, Y

Zhao, Y Wu, Y Wang, J

Permalink

Publisher:: Elsevier
Publication Type:: Journal Article
Citation:: Journal of Systems and Software, 2017, 125, pp. 35-46
Issue Date:: 2017-03-01

Closed Access

	Filename	Description	Size
	1-s2.0-S0164121216302400-main.pdf		1.33 MB	Adobe PDF	View/Open

Copyright Clearance Process

Recently Added
In Progress
Closed Access

This item is closed access and not available.

Full metadata record

Field	Value	Language
dc.contributor.author	Ma, J
dc.contributor.author	Chen, L
dc.contributor.author	Lv, M
dc.contributor.author	Yang, Y https://orcid.org/0000-0002-0512-880X
dc.contributor.author	Zhao, Y
dc.contributor.author	Wu, Y
dc.contributor.author	Wang, J
dc.date.accessioned	2022-08-01T05:43:24Z
dc.date.available	2022-08-01T05:43:24Z
dc.date.issued	2017-03-01
dc.identifier.citation	Journal of Systems and Software, 2017, 125, pp. 35-46
dc.identifier.issn	0164-1212
dc.identifier.issn	1873-1228
dc.identifier.uri	http://hdl.handle.net/10453/159435
dc.description.abstract	Cloudera Impala, an analytic database system for Apache Hadoop, has a severe problem with query plan generation: the system can only generate query plans in left-deep tree form, which restricts the ability of parallel execution. In this paper, we present a logical query optimization scheme for Impala system. First, an improved McCHyp (MinCutConservative Hypergraph) logical query plan generation algorithm is proposed for Impala system. It can reduce the plan generation time by introducing a pruning strategy. Second, a new cost model that takes the characteristics of Impala system into account is proposed. Finally, Impala system is extended to support query plans in bushy tree form by integrating the plan generation algorithm. We evaluated our scheme using TPC-DS test suit. Experimental results show that the extended Impala system generally performs better than the original system, and the improved plan generation algorithm has less execution time than McCHyp. In addition, our cost model fits better for Impala system, which supports query plans in bushy tree form.
dc.language	English
dc.publisher	Elsevier
dc.relation.ispartof	Journal of Systems and Software
dc.relation.isbasedon	10.1016/j.jss.2016.11.038
dc.rights	info:eu-repo/semantics/closedAccess
dc.subject	0803 Computer Software, 0804 Data Format, 0806 Information Systems
dc.subject.classification	Software Engineering
dc.title	Logical query optimization for Cloudera Impala system
dc.type	Journal Article
utslib.citation.volume	125
utslib.for	0803 Computer Software
utslib.for	0804 Data Format
utslib.for	0806 Information Systems
pubs.organisational-group	/University of Technology Sydney
pubs.organisational-group	/University of Technology Sydney/Faculty of Engineering and Information Technology
utslib.copyright.status	closed_access	*
pubs.consider-herdc	false
dc.date.updated	2022-08-01T05:43:22Z
pubs.publication-status	Published
pubs.volume	125

Abstract:

Cloudera Impala, an analytic database system for Apache Hadoop, has a severe problem with query plan generation: the system can only generate query plans in left-deep tree form, which restricts the ability of parallel execution. In this paper, we present a logical query optimization scheme for Impala system. First, an improved McCHyp (MinCutConservative Hypergraph) logical query plan generation algorithm is proposed for Impala system. It can reduce the plan generation time by introducing a pruning strategy. Second, a new cost model that takes the characteristics of Impala system into account is proposed. Finally, Impala system is extended to support query plans in bushy tree form by integrating the plan generation algorithm. We evaluated our scheme using TPC-DS test suit. Experimental results show that the extended Impala system generally performs better than the original system, and the improved plan generation algorithm has less execution time than McCHyp. In addition, our cost model fits better for Impala system, which supports query plans in bushy tree form.

Please use this identifier to cite or link to this item:

http://hdl.handle.net/10453/159435