An Integrated Approach for Massive Sequential Data Processing in Civil Infrastructure Operation and Maintenance

Yu, G; Liu, J; Du, J; Hu, M; Sugumaran, V

An Integrated Approach for Massive Sequential Data Processing in Civil Infrastructure Operation and Maintenance

Yu, G Liu, J Du, J

Hu, M Sugumaran, V

Permalink

Publisher:: IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
Publication Type:: Journal Article
Citation:: IEEE Access, 2018, 6, pp. 19739-19751
Issue Date:: 2018-03-14

Closed Access

	Filename	Description	Size
	An_Integrated_Approach_for_Massive_Sequential_Data_Processing_in_Civil_Infrastructure_Operation_and_Maintenance.pdf	Published version	6.1 MB	Adobe PDF	View/Open

Copyright Clearance Process

Recently Added
In Progress
Closed Access

This item is closed access and not available.

Full metadata record

Field	Value	Language
dc.contributor.author	Yu, G
dc.contributor.author	Liu, J
dc.contributor.author	Du, J https://orcid.org/0000-0002-6428-7763
dc.contributor.author	Hu, M
dc.contributor.author	Sugumaran, V
dc.date.accessioned	2022-08-25T19:23:22Z
dc.date.available	2022-08-25T19:23:22Z
dc.date.issued	2018-03-14
dc.identifier.citation	IEEE Access, 2018, 6, pp. 19739-19751
dc.identifier.issn	2169-3536
dc.identifier.issn	2169-3536
dc.identifier.uri	http://hdl.handle.net/10453/160869
dc.description.abstract	This paper presents an extract-transform-load (ETL) approach based on multilayer task execution for processing massive sequential data collected from infrastructure operation and maintenance. The proposed approach consists of ETL task partition, execution mode selection, and ETL modeling. The task partition focuses on dividing the ETL process into four tasks to be executed in accordance with different organizational forms of data. Sequenced or non-sequenced load mode is optional, which is independent of the data standardization. In addition, the ETL modeling phase implements conceptual, logical, and physical modeling for the multi-dimensional model. Our main objective is to integrate massive sequential data, enhancing decision-making performance for the intelligent management platform. Traffic data for two years were collected from various systems and acquisition tools of different providers to evaluate the data integration capability of the proposed approach. Furthermore, Kettle software was used to perform transformation and job modules for the multilayer tasks. In addition, a machine learning algorithm was used to generate traffic warning in the tunnels based on the integrated data. The proposed approach is promising for the management and analysis of massive sequential data generated in operation, the maintenance of transportation tunnels, and effiective decision-making.
dc.language	English
dc.publisher	IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
dc.relation.ispartof	IEEE Access
dc.relation.isbasedon	10.1109/ACCESS.2018.2816001
dc.rights	info:eu-repo/semantics/closedAccess
dc.subject	08 Information and Computing Sciences, 09 Engineering, 10 Technology
dc.title	An Integrated Approach for Massive Sequential Data Processing in Civil Infrastructure Operation and Maintenance
dc.type	Journal Article
utslib.citation.volume	6
utslib.for	08 Information and Computing Sciences
utslib.for	09 Engineering
utslib.for	10 Technology
pubs.organisational-group	/University of Technology Sydney
pubs.organisational-group	/University of Technology Sydney/Faculty of Engineering and Information Technology
utslib.copyright.status	closed_access	*
dc.date.updated	2022-08-25T19:23:17Z
pubs.publication-status	Published
pubs.volume	6

Abstract:

This paper presents an extract-transform-load (ETL) approach based on multilayer task execution for processing massive sequential data collected from infrastructure operation and maintenance. The proposed approach consists of ETL task partition, execution mode selection, and ETL modeling. The task partition focuses on dividing the ETL process into four tasks to be executed in accordance with different organizational forms of data. Sequenced or non-sequenced load mode is optional, which is independent of the data standardization. In addition, the ETL modeling phase implements conceptual, logical, and physical modeling for the multi-dimensional model. Our main objective is to integrate massive sequential data, enhancing decision-making performance for the intelligent management platform. Traffic data for two years were collected from various systems and acquisition tools of different providers to evaluate the data integration capability of the proposed approach. Furthermore, Kettle software was used to perform transformation and job modules for the multilayer tasks. In addition, a machine learning algorithm was used to generate traffic warning in the tunnels based on the integrated data. The proposed approach is promising for the management and analysis of massive sequential data generated in operation, the maintenance of transportation tunnels, and effiective decision-making.

Please use this identifier to cite or link to this item:

http://hdl.handle.net/10453/160869