Cloud-assisted multiview video summarization using CNN and bidirectional LSTM

Hussain, T; Muhammad, K; Ullah, A; Cao, Z; Baik, SW; De Albuquerque, VHC

Cloud-assisted multiview video summarization using CNN and bidirectional LSTM

Hussain, T Muhammad, K Ullah, A Cao, Z

Baik, SW De Albuquerque, VHC

Permalink

Publisher:: IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
Publication Type:: Journal Article
Citation:: IEEE Transactions on Industrial Informatics, 2020, 16, (1), pp. 77-86
Issue Date:: 2020-01-01

Closed Access

	Filename	Description	Size
	08765236.pdf	Published version	4.86 MB	Adobe PDF	View/Open

Copyright Clearance Process

Recently Added
In Progress
Closed Access

This item is closed access and not available.

Full metadata record

Field	Value	Language
dc.contributor.author	Hussain, T
dc.contributor.author	Muhammad, K
dc.contributor.author	Ullah, A
dc.contributor.author	Cao, Z https://orcid.org/0000-0003-3656-0328
dc.contributor.author	Baik, SW
dc.contributor.author	De Albuquerque, VHC
dc.date.accessioned	2020-11-09T02:55:24Z
dc.date.available	2020-11-09T02:55:24Z
dc.date.issued	2020-01-01
dc.identifier.citation	IEEE Transactions on Industrial Informatics, 2020, 16, (1), pp. 77-86
dc.identifier.issn	1551-3203
dc.identifier.issn	1941-0050
dc.identifier.uri	http://hdl.handle.net/10453/143858
dc.description.abstract	© 2005-2012 IEEE. The massive amount of video data produced by surveillance networks in industries instigate various challenges in exploring these videos for many applications, such as video summarization (VS), analysis, indexing, and retrieval. The task of multiview video summarization (MVS) is very challenging due to the gigantic size of data, redundancy, overlapping in views, light variations, and interview correlations. To address these challenges, various low-level features and clustering-based soft computing techniques are proposed that cannot fully exploit MVS. In this article, we achieve MVS by integrating deep neural network based soft computing techniques in a two-tier framework. The first online tier performs target-appearance-based shots segmentation and stores them in a lookup table that is transmitted to cloud for further processing. The second tier extracts deep features from each frame of a sequence in the lookup table and pass them to deep bidirectional long short-term memory (DB-LSTM) to acquire probabilities of informativeness and generates a summary. Experimental evaluation on benchmark dataset and industrial surveillance data from YouTube confirms the better performance of our system compared to the state-of-the-art MVS methods.
dc.language	English
dc.publisher	IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
dc.relation.ispartof	IEEE Transactions on Industrial Informatics
dc.relation.isbasedon	10.1109/TII.2019.2929228
dc.rights	© 2020 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.	en_US
dc.rights	info:eu-repo/semantics/closedAccess
dc.subject	08 Information and Computing Sciences, 09 Engineering, 10 Technology
dc.subject.classification	Electrical & Electronic Engineering
dc.title	Cloud-assisted multiview video summarization using CNN and bidirectional LSTM
dc.type	Journal Article
utslib.citation.volume	16
utslib.for	08 Information and Computing Sciences
utslib.for	09 Engineering
utslib.for	10 Technology
pubs.organisational-group	/University of Technology Sydney/Faculty of Engineering and Information Technology
pubs.organisational-group	/University of Technology Sydney/Faculty of Engineering and Information Technology/School of Computer Science
pubs.organisational-group	/University of Technology Sydney
utslib.copyright.status	closed_access	*
dc.date.updated	2020-11-09T02:55:17Z
pubs.issue	1
pubs.publication-status	Published
pubs.volume	16
utslib.citation.issue	1

Abstract:

© 2005-2012 IEEE. The massive amount of video data produced by surveillance networks in industries instigate various challenges in exploring these videos for many applications, such as video summarization (VS), analysis, indexing, and retrieval. The task of multiview video summarization (MVS) is very challenging due to the gigantic size of data, redundancy, overlapping in views, light variations, and interview correlations. To address these challenges, various low-level features and clustering-based soft computing techniques are proposed that cannot fully exploit MVS. In this article, we achieve MVS by integrating deep neural network based soft computing techniques in a two-tier framework. The first online tier performs target-appearance-based shots segmentation and stores them in a lookup table that is transmitted to cloud for further processing. The second tier extracts deep features from each frame of a sequence in the lookup table and pass them to deep bidirectional long short-term memory (DB-LSTM) to acquire probabilities of informativeness and generates a summary. Experimental evaluation on benchmark dataset and industrial surveillance data from YouTube confirms the better performance of our system compared to the state-of-the-art MVS methods.

Please use this identifier to cite or link to this item:

http://hdl.handle.net/10453/143858