Augmenting image descriptions using structured prediction output

Publication Type:
Journal Article
IEEE Transactions on Multimedia, 2014, 16 (6), pp. 1665 - 1676
Issue Date:
Filename Description Size
Augmenting Image Descriptions Using.pdfPublished Version2.78 MB
Adobe PDF
Full metadata record
© 2014 IEEE. The need for richer descriptions of images arises in a wide spectrum of applications ranging from image understanding to image retrieval. While the Automatic Image Annotation (AIA) has been extensively studied, image descriptions with the output labels lack sufficient information. This paper proposes to augment image descriptions using structured prediction output. We define a hierarchical tree-structured semantic unit to describe images, from which we can obtain not only the class and subclass one image belongs to, but also the attributes one image has. After defining a new feature map function of structured SVM, we decompose the loss function into every node of the hierarchical tree-structured semantic unit and then predict the tree-structured semantic unit for testing images. In the experiments, we evaluate the performance of the proposed method on two open benchmark datasets and compare with the state-of-the-art methods. Experimental results show the better prediction performance of the proposed method and demonstrate the strength of augmenting image descriptions.
Please use this identifier to cite or link to this item: