Modeling Multiple Aesthetic Views for Series Photo Selection

Huang, J; Gong, Y; Zhang, L; Zhang, J; Nie, L; Yin, Y

Modeling Multiple Aesthetic Views for Series Photo Selection

Huang, J Gong, Y Zhang, L Zhang, J

Nie, L Yin, Y

Permalink

Publisher:: Institute of Electrical and Electronics Engineers (IEEE)
Publication Type:: Journal Article
Citation:: IEEE Transactions on Multimedia, 2023, PP, (99), pp. 1-14
Issue Date:: 2023-01-01

Closed Access

	Filename	Description	Size
	Modeling_Multiple_Aesthetic_Views_for_Series_Photo_Selection.pdf	Published version	3.6 MB	Adobe PDF	View/Open

Copyright Clearance Process

Recently Added
In Progress
Closed Access

This item is closed access and not available.

Full metadata record

Field	Value	Language
dc.contributor.author	Huang, J
dc.contributor.author	Gong, Y
dc.contributor.author	Zhang, L
dc.contributor.author	Zhang, J https://orcid.org/0000-0002-7240-3541
dc.contributor.author	Nie, L
dc.contributor.author	Yin, Y
dc.date.accessioned	2024-01-29T04:18:10Z
dc.date.available	2024-01-29T04:18:10Z
dc.date.issued	2023-01-01
dc.identifier.citation	IEEE Transactions on Multimedia, 2023, PP, (99), pp. 1-14
dc.identifier.issn	1520-9210
dc.identifier.issn	1941-0077
dc.identifier.uri	http://hdl.handle.net/10453/175005
dc.description.abstract	Numerous photos are taken in daily life, and sorting them is laborious and time consuming. The large number of similar images exacerbates the difficulty of album management, under this scenario, serial photo selection (SPS) emerges. As an important branch of image aesthetic quality assessment, it focuses on identifying the best image among a series of almost identical photos. Currently, most existing SPS methods focus only on extracting features from the original image, while neglecting the fact that multiple views of the image can provide much more detailed aesthetic information. In this paper, we propose a Siamese network structure called SPSNet to enhance the representation learning of multi-view features by acquiring the depth, generic, and handcrafted features of images. In specific, we implement a parallel structure to extract deep and shallow features, fusing local and global representations at different resolutions interactively. The aggregation of multiple views of image via a self-attentive module with adaptive weights enables the model to discriminate the importance of each view. Moreover, we employ a graph neural network to construct the relationships among the multi-view features. Our proposed method, which is trained by a Siamese network, can effectively distinguish the nuances of similar images, and thus, select the best one from a series of almost identical photos. Extensive experiments conducted on the aesthetic dataset demonstrate that our method outperforms other state-of-the-art SPS methods, which achieves the 75.36% accuracy on the Phototriage dataset. Besides, our model is up to 3.04% better than the baseline methods in terms of the average accuracy.
dc.language	en
dc.publisher	Institute of Electrical and Electronics Engineers (IEEE)
dc.relation.ispartof	IEEE Transactions on Multimedia
dc.relation.isbasedon	10.1109/TMM.2023.3290751
dc.rights	info:eu-repo/semantics/closedAccess
dc.subject	08 Information and Computing Sciences, 09 Engineering
dc.subject.classification	Artificial Intelligence & Image Processing
dc.subject.classification	40 Engineering
dc.subject.classification	46 Information and computing sciences
dc.title	Modeling Multiple Aesthetic Views for Series Photo Selection
dc.type	Journal Article
utslib.citation.volume	PP
utslib.for	08 Information and Computing Sciences
utslib.for	09 Engineering
pubs.organisational-group	University of Technology Sydney
pubs.organisational-group	University of Technology Sydney/Faculty of Engineering and Information Technology
pubs.organisational-group	University of Technology Sydney/Strength - GBDTC - Global Big Data Technologies
pubs.organisational-group	University of Technology Sydney/Faculty of Engineering and Information Technology/School of Electrical and Data Engineering
utslib.copyright.status	closed_access	*
dc.date.updated	2024-01-29T04:18:04Z
pubs.issue	99
pubs.publication-status	Published
pubs.volume	PP
utslib.citation.issue	99

Abstract:

Numerous photos are taken in daily life, and sorting them is laborious and time consuming. The large number of similar images exacerbates the difficulty of album management, under this scenario, serial photo selection (SPS) emerges. As an important branch of image aesthetic quality assessment, it focuses on identifying the best image among a series of almost identical photos. Currently, most existing SPS methods focus only on extracting features from the original image, while neglecting the fact that multiple views of the image can provide much more detailed aesthetic information. In this paper, we propose a Siamese network structure called SPSNet to enhance the representation learning of multi-view features by acquiring the depth, generic, and handcrafted features of images. In specific, we implement a parallel structure to extract deep and shallow features, fusing local and global representations at different resolutions interactively. The aggregation of multiple views of image via a self-attentive module with adaptive weights enables the model to discriminate the importance of each view. Moreover, we employ a graph neural network to construct the relationships among the multi-view features. Our proposed method, which is trained by a Siamese network, can effectively distinguish the nuances of similar images, and thus, select the best one from a series of almost identical photos. Extensive experiments conducted on the aesthetic dataset demonstrate that our method outperforms other state-of-the-art SPS methods, which achieves the 75.36% accuracy on the Phototriage dataset. Besides, our model is up to 3.04% better than the baseline methods in terms of the average accuracy.

Please use this identifier to cite or link to this item:

http://hdl.handle.net/10453/175005