Augmenting image descriptions using structured prediction output

Han, Y; Wei, X; Cao, X; Yang, Y; Zhou, X

Augmenting image descriptions using structured prediction output

Han, Y Wei, X Cao, X Yang, Y

Zhou, X

Permalink

Publication Type:: Journal Article
Citation:: IEEE Transactions on Multimedia, 2014, 16 (6), pp. 1665 - 1676
Issue Date:: 2014-10-01

Closed Access

	Filename	Description	Size
	Augmenting Image Descriptions Using.pdf	Published Version	2.78 MB	Adobe PDF	View/Open

Copyright Clearance Process

Recently Added
In Progress
Closed Access

This item is closed access and not available.

Full metadata record

Field	Value	Language
dc.contributor.author	Han, Y	en_US
dc.contributor.author	Wei, X	en_US
dc.contributor.author	Cao, X	en_US
dc.contributor.author	Yang, Y https://orcid.org/0000-0001-5528-0546	en_US
dc.contributor.author	Zhou, X	en_US
dc.date.issued	2014-10-01	en_US
dc.identifier.citation	IEEE Transactions on Multimedia, 2014, 16 (6), pp. 1665 - 1676	en_US
dc.identifier.issn	1520-9210	en_US
dc.identifier.uri	http://hdl.handle.net/10453/115888
dc.description.abstract	© 2014 IEEE. The need for richer descriptions of images arises in a wide spectrum of applications ranging from image understanding to image retrieval. While the Automatic Image Annotation (AIA) has been extensively studied, image descriptions with the output labels lack sufficient information. This paper proposes to augment image descriptions using structured prediction output. We define a hierarchical tree-structured semantic unit to describe images, from which we can obtain not only the class and subclass one image belongs to, but also the attributes one image has. After defining a new feature map function of structured SVM, we decompose the loss function into every node of the hierarchical tree-structured semantic unit and then predict the tree-structured semantic unit for testing images. In the experiments, we evaluate the performance of the proposed method on two open benchmark datasets and compare with the state-of-the-art methods. Experimental results show the better prediction performance of the proposed method and demonstrate the strength of augmenting image descriptions.	en_US
dc.relation.ispartof	IEEE Transactions on Multimedia	en_US
dc.relation.isbasedon	10.1109/TMM.2014.2321530	en_US
dc.subject.classification	Artificial Intelligence & Image Processing	en_US
dc.title	Augmenting image descriptions using structured prediction output	en_US
dc.type	Journal Article
utslib.citation.volume	6	en_US
utslib.citation.volume	16	en_US
utslib.for	0801 Artificial Intelligence and Image Processing	en_US
utslib.for	08 Information and Computing Sciences	en_US
utslib.for	09 Engineering	en_US
pubs.embargo.period	Not known	en_US
pubs.organisational-group	/University of Technology Sydney
pubs.organisational-group	/University of Technology Sydney/Faculty of Engineering and Information Technology
pubs.organisational-group	/University of Technology Sydney/Strength - CAI - Centre for Artificial Intelligence
utslib.copyright.status	closed_access
pubs.issue	6	en_US
pubs.publication-status	Published	en_US
pubs.volume	16	en_US

Abstract:

© 2014 IEEE. The need for richer descriptions of images arises in a wide spectrum of applications ranging from image understanding to image retrieval. While the Automatic Image Annotation (AIA) has been extensively studied, image descriptions with the output labels lack sufficient information. This paper proposes to augment image descriptions using structured prediction output. We define a hierarchical tree-structured semantic unit to describe images, from which we can obtain not only the class and subclass one image belongs to, but also the attributes one image has. After defining a new feature map function of structured SVM, we decompose the loss function into every node of the hierarchical tree-structured semantic unit and then predict the tree-structured semantic unit for testing images. In the experiments, we evaluate the performance of the proposed method on two open benchmark datasets and compare with the state-of-the-art methods. Experimental results show the better prediction performance of the proposed method and demonstrate the strength of augmenting image descriptions.

Please use this identifier to cite or link to this item:

http://hdl.handle.net/10453/115888