Fast human action classification and VOI localization with enhanced sparse coding

Lu, S; Zhang, J; Wang, Z; Feng, DD

Fast human action classification and VOI localization with enhanced sparse coding

Lu, S Zhang, J

Wang, Z Feng, DD

Permalink

Publication Type:: Journal Article
Citation:: Journal of Visual Communication and Image Representation, 2013, 24 (2), pp. 127 - 136
Issue Date:: 2013-01-01

Closed Access

	Filename	Description	Size
	2012004047OK.pdf		1.07 MB	Adobe PDF	View/Open

Copyright Clearance Process

Recently Added
In Progress
Closed Access

This item is closed access and not available.

Full metadata record

Field	Value	Language
dc.contributor.author	Lu, S	en_US
dc.contributor.author	Zhang, J https://orcid.org/0000-0002-7240-3541	en_US
dc.contributor.author	Wang, Z	en_US
dc.contributor.author	Feng, DD	en_US
dc.date.issued	2013-01-01	en_US
dc.identifier.citation	Journal of Visual Communication and Image Representation, 2013, 24 (2), pp. 127 - 136	en_US
dc.identifier.issn	1047-3203	en_US
dc.identifier.uri	http://hdl.handle.net/10453/27489
dc.description.abstract	Sparse coding which encodes the natural visual signal into a sparse space for visual codebook generation and feature quantization, has been successfully utilized for many image classification applications. However, it has been seldom explored for many video analysis tasks. In particular, the increased complexity in characterizing the visual patterns of diverse human actions with both the spatial and temporal variations imposes more challenges to the conventional sparse coding scheme. In this paper, we propose an enhanced sparse coding scheme through learning discriminative dictionary and optimizing the local pooling strategy. Localizing when and where a specific action happens in realistic videos is another challenging task. By utilizing the sparse coding based representations of human actions, this paper further presents a novel coarse-to-fine framework to localize the Volumes of Interest (VOIs) for the actions. Firstly, local visual features are transformed into the sparse signal domain through our enhanced sparse coding scheme. Secondly, in order to avoid exhaustive scan of entire videos for the VOI localization, we extend the Spatial Pyramid Matching into temporal domain, namely Spatial Temporal Pyramid Matching, to obtain the VOI candidates. Finally, a multi-level branch-and-bound approach is developed to refine the VOI candidates. The proposed framework is also able to avoid prohibitive computations in local similarity matching (e.g., nearest neighbors voting). Experimental results on both two popular benchmark datasets (KTH and YouTube UCF) and the widely used localization dataset (MSR) demonstrate that our approach reduces computational cost significantly while maintaining comparable classification accuracy to that of the state-of-the-art methods. © 2012 Elsevier Inc. All rights reserved.	en_US
dc.relation.ispartof	Journal of Visual Communication and Image Representation	en_US
dc.relation.isbasedon	10.1016/j.jvcir.2012.07.008	en_US
dc.subject.classification	Artificial Intelligence & Image Processing	en_US
dc.title	Fast human action classification and VOI localization with enhanced sparse coding	en_US
dc.type	Journal Article
utslib.citation.volume	2	en_US
utslib.citation.volume	24	en_US
utslib.for	0801 Artificial Intelligence and Image Processing	en_US
utslib.for	1203 Design Practice and Management	en_US
utslib.for	1905 Visual Arts and Crafts	en_US
pubs.embargo.period	Not known	en_US
pubs.organisational-group	/University of Technology Sydney
pubs.organisational-group	/University of Technology Sydney/Faculty of Engineering and Information Technology
pubs.organisational-group	/University of Technology Sydney/Faculty of Engineering and Information Technology/School of Electrical and Data Engineering
pubs.organisational-group	/University of Technology Sydney/Strength - GBDTC - Global Big Data Technologies
utslib.copyright.status	closed_access
pubs.issue	2	en_US
pubs.publication-status	Published	en_US
pubs.volume	24	en_US

Abstract:

Sparse coding which encodes the natural visual signal into a sparse space for visual codebook generation and feature quantization, has been successfully utilized for many image classification applications. However, it has been seldom explored for many video analysis tasks. In particular, the increased complexity in characterizing the visual patterns of diverse human actions with both the spatial and temporal variations imposes more challenges to the conventional sparse coding scheme. In this paper, we propose an enhanced sparse coding scheme through learning discriminative dictionary and optimizing the local pooling strategy. Localizing when and where a specific action happens in realistic videos is another challenging task. By utilizing the sparse coding based representations of human actions, this paper further presents a novel coarse-to-fine framework to localize the Volumes of Interest (VOIs) for the actions. Firstly, local visual features are transformed into the sparse signal domain through our enhanced sparse coding scheme. Secondly, in order to avoid exhaustive scan of entire videos for the VOI localization, we extend the Spatial Pyramid Matching into temporal domain, namely Spatial Temporal Pyramid Matching, to obtain the VOI candidates. Finally, a multi-level branch-and-bound approach is developed to refine the VOI candidates. The proposed framework is also able to avoid prohibitive computations in local similarity matching (e.g., nearest neighbors voting). Experimental results on both two popular benchmark datasets (KTH and YouTube UCF) and the widely used localization dataset (MSR) demonstrate that our approach reduces computational cost significantly while maintaining comparable classification accuracy to that of the state-of-the-art methods. © 2012 Elsevier Inc. All rights reserved.

Please use this identifier to cite or link to this item:

http://hdl.handle.net/10453/27489