Memory-Based Network for Scene Graph with Unbalanced Relations

Wang, W; Liu, R; Wang, M; Wang, S; Chang, X; Chen, Y

Memory-Based Network for Scene Graph with Unbalanced Relations

Wang, W Liu, R Wang, M Wang, S Chang, X

Chen, Y

Permalink

Publisher:: ASSOC COMPUTING MACHINERY
Publication Type:: Conference Proceeding
Citation:: MM 2020 - Proceedings of the 28th ACM International Conference on Multimedia, 2020, pp. 2400-2408
Issue Date:: 2020-10-12

Closed Access

	Filename	Description	Size
	3394171.3413507.pdf	Accepted version	3.84 MB	Adobe PDF	View/Open

Copyright Clearance Process

Recently Added
In Progress
Closed Access

This item is closed access and not available.

Full metadata record

Field	Value	Language
dc.contributor.author	Wang, W
dc.contributor.author	Liu, R
dc.contributor.author	Wang, M
dc.contributor.author	Wang, S
dc.contributor.author	Chang, X https://orcid.org/0000-0002-7778-8807
dc.contributor.author	Chen, Y
dc.date	2020-10-12
dc.date.accessioned	2023-03-31T10:16:12Z
dc.date.available	2023-03-31T10:16:12Z
dc.date.issued	2020-10-12
dc.identifier.citation	MM 2020 - Proceedings of the 28th ACM International Conference on Multimedia, 2020, pp. 2400-2408
dc.identifier.isbn	9781450379885
dc.identifier.uri	http://hdl.handle.net/10453/168978
dc.description.abstract	The scene graph which can be represented by a set of visual triples is composed of objects and the relations between object pairs. It is vital for image captioning, visual question answering, and many other applications. However, there is a long tail distribution on the scene graph dataset, and the tail relation cannot be accurately identified due to the lack of training samples. The problem of the nonstandard label and feature overlap on the scene graph affects the extraction of discriminative features and exacerbates the effect of data imbalance on the model. For these reasons, we propose a novel scene graph generation model that can effectively improve the detection of low-frequency relations. We use the method of memory features to realize the transfer of high-frequency relation features to low-frequency relation features. Extensive experiments on scene graph datasets show that our model significantly improved the performance of two evaluation metrics R@K and mR@K compared with state-of-the-art baselines.
dc.language	en
dc.publisher	ASSOC COMPUTING MACHINERY
dc.relation.ispartof	MM 2020 - Proceedings of the 28th ACM International Conference on Multimedia
dc.relation.ispartof	28th ACM International Conference on Multimedia (MM)
dc.relation.isbasedon	10.1145/3394171.3413507
dc.rights	info:eu-repo/semantics/closedAccess
dc.title	Memory-Based Network for Scene Graph with Unbalanced Relations
dc.type	Conference Proceeding
utslib.location.activity	ELECTR NETWORK
pubs.organisational-group	/University of Technology Sydney
pubs.organisational-group	/University of Technology Sydney/Faculty of Engineering and Information Technology
pubs.organisational-group	/University of Technology Sydney/Strength - AAII - Australian Artificial Intelligence Institute
utslib.copyright.status	closed_access	*
dc.date.updated	2023-03-31T10:16:09Z
pubs.finish-date	2020-10-16
pubs.publication-status	Published
pubs.start-date	2020-10-12

Abstract:

The scene graph which can be represented by a set of visual triples is composed of objects and the relations between object pairs. It is vital for image captioning, visual question answering, and many other applications. However, there is a long tail distribution on the scene graph dataset, and the tail relation cannot be accurately identified due to the lack of training samples. The problem of the nonstandard label and feature overlap on the scene graph affects the extraction of discriminative features and exacerbates the effect of data imbalance on the model. For these reasons, we propose a novel scene graph generation model that can effectively improve the detection of low-frequency relations. We use the method of memory features to realize the transfer of high-frequency relation features to low-frequency relation features. Extensive experiments on scene graph datasets show that our model significantly improved the performance of two evaluation metrics R@K and mR@K compared with state-of-the-art baselines.

Please use this identifier to cite or link to this item:

http://hdl.handle.net/10453/168978