GCM-Net: Towards Effective Global Context Modeling for Image Inpainting

Zheng, H; Zhang, Z; Wang, Y; Zhang, Z; Xu, M; Yang, Y; Wang, M

GCM-Net: Towards Effective Global Context Modeling for Image Inpainting

Zheng, H Zhang, Z Wang, Y Zhang, Z Xu, M Yang, Y

Wang, M

Permalink

Publisher:: ACM
Publication Type:: Conference Proceeding
Citation:: MM 2021 - Proceedings of the 29th ACM International Conference on Multimedia, 2021, pp. 2586-2594
Issue Date:: 2021-10-17

Closed Access

	Filename	Description	Size
	3474085.3475433_published.pdf	Published version	1.83 MB	Adobe PDF	View/Open

Copyright Clearance Process

Recently Added
In Progress
Closed Access

This item is closed access and not available.

Full metadata record

Field	Value	Language
dc.contributor.author	Zheng, H
dc.contributor.author	Zhang, Z
dc.contributor.author	Wang, Y
dc.contributor.author	Zhang, Z
dc.contributor.author	Xu, M
dc.contributor.author	Yang, Y https://orcid.org/0000-0002-0512-880X
dc.contributor.author	Wang, M
dc.date.accessioned	2022-03-31T06:16:22Z
dc.date.available	2022-03-31T06:16:22Z
dc.date.issued	2021-10-17
dc.identifier.citation	MM 2021 - Proceedings of the 29th ACM International Conference on Multimedia, 2021, pp. 2586-2594
dc.identifier.isbn	9781450386517
dc.identifier.uri	http://hdl.handle.net/10453/155787
dc.description.abstract	Deep learning based inpainting methods have obtained promising performance for image restoration, however current image inpainting methods still tend to produce unreasonable structures and blurry textures when processing the damaged images with heavy corruptions. In this paper, we propose a new image inpainting method termed Global Context Modeling Network (GCM-Net). By capturing the global contextual information, GCM-Net can potentially improve the performance of recovering the missing region in the damaged images with irregular masks. To be specific, we first use four convolution layers to extract the shadow features. Then, we design a progressive multi-scale fusion block termed PMSFB to extract and fuse the multi-scale features for obtaining local features. Besides, a dense context extraction (DCE) module is also designed to aggregate the local features extracted by PMSFBs. To improve the information flow, a channel attention guided residual learning module is deployed in both the DCE and PMSFB, which can reweight the learned residual features and refine the extracted information. To capture more global contextual information and enhance the representation ability, a coordinate context attention (CCA) based module is also presented. Finally, the extracted features with rich information are decoded as the image inpainting result. Extensive results on the Paris Street View, Places2 and CelebA-HQ datasets demonstrate that our method can better recover the structures and textures, and deliver significant improvements, compared with some related inpainting methods.
dc.language	en
dc.publisher	ACM
dc.relation.ispartof	MM 2021 - Proceedings of the 29th ACM International Conference on Multimedia
dc.relation.ispartof	Proceedings of the 29th ACM International Conference on Multimedia
dc.relation.isbasedon	10.1145/3474085.3475433
dc.rights	info:eu-repo/semantics/closedAccess
dc.title	GCM-Net: Towards Effective Global Context Modeling for Image Inpainting
dc.type	Conference Proceeding
pubs.organisational-group	/University of Technology Sydney
pubs.organisational-group	/University of Technology Sydney/Faculty of Engineering and Information Technology
pubs.organisational-group	/University of Technology Sydney/Strength - AAII - Australian Artificial Intelligence Institute
utslib.copyright.status	closed_access	*
dc.date.updated	2022-03-31T06:16:20Z
pubs.publication-status	Published

Abstract:

Deep learning based inpainting methods have obtained promising performance for image restoration, however current image inpainting methods still tend to produce unreasonable structures and blurry textures when processing the damaged images with heavy corruptions. In this paper, we propose a new image inpainting method termed Global Context Modeling Network (GCM-Net). By capturing the global contextual information, GCM-Net can potentially improve the performance of recovering the missing region in the damaged images with irregular masks. To be specific, we first use four convolution layers to extract the shadow features. Then, we design a progressive multi-scale fusion block termed PMSFB to extract and fuse the multi-scale features for obtaining local features. Besides, a dense context extraction (DCE) module is also designed to aggregate the local features extracted by PMSFBs. To improve the information flow, a channel attention guided residual learning module is deployed in both the DCE and PMSFB, which can reweight the learned residual features and refine the extracted information. To capture more global contextual information and enhance the representation ability, a coordinate context attention (CCA) based module is also presented. Finally, the extracted features with rich information are decoded as the image inpainting result. Extensive results on the Paris Street View, Places2 and CelebA-HQ datasets demonstrate that our method can better recover the structures and textures, and deliver significant improvements, compared with some related inpainting methods.

Please use this identifier to cite or link to this item:

http://hdl.handle.net/10453/155787