Deep Multi-Resolution Mutual Learning for Image Inpainting

Zheng, H; Zhang, Z; Zhang, H; Yang, Y; Yan, S; Wang, M

Deep Multi-Resolution Mutual Learning for Image Inpainting

Zheng, H Zhang, Z Zhang, H Yang, Y

Yan, S Wang, M

Permalink

Publisher:: Association for Computing Machinery (ACM)
Publication Type:: Conference Proceeding
Citation:: MM 2022 - Proceedings of the 30th ACM International Conference on Multimedia, 2022, pp. 6359-6367
Issue Date:: 2022-10-10

Closed Access

	Filename	Description	Size
	Deep Multi-Resolution Mutual Learning for Image Inpainting.pdf	Published version	6.06 MB	Adobe PDF	View/Open

Copyright Clearance Process

Recently Added
In Progress
Closed Access

This item is closed access and not available.

Full metadata record

Field	Value	Language
dc.contributor.author	Zheng, H
dc.contributor.author	Zhang, Z
dc.contributor.author	Zhang, H
dc.contributor.author	Yang, Y https://orcid.org/0000-0002-0512-880X
dc.contributor.author	Yan, S
dc.contributor.author	Wang, M
dc.date.accessioned	2023-05-19T04:49:52Z
dc.date.available	2023-05-19T04:49:52Z
dc.date.issued	2022-10-10
dc.identifier.citation	MM 2022 - Proceedings of the 30th ACM International Conference on Multimedia, 2022, pp. 6359-6367
dc.identifier.isbn	9781450392037
dc.identifier.uri	http://hdl.handle.net/10453/170372
dc.description.abstract	Deep image inpainting methods have improved the inpainting performance greatly due to the powerful representation ability of deep learning. However, current deep inpainting networks still tend to produce unreasonable structures and blurry textures due to the ill-posed properties of the task, i.e., image inpainting is still a challenging topic. In this paper, we therefore propose a novel deep multi-resolution mutual learning (DMRML) strategy, which can fully explore the information from various resolutions. Specifically, we design a new image inpainting network, termed multi-resolution mutual network (MRM-Net), which takes the damaged images of different resolutions as input, then excavates and exploits the correlation among different resolutions to guide the image inpainting process. Technically, we designs two new modules called multi-resolution information interaction (MRII) and adaptive content enhancement (ACE). MRII aims at discovering the correlation of multiple resolutions and exchanging information, and ACE focuses on enhancing the contents using the interacted features. Note that we also present an memory preservation mechanism (MPM) to prevent from the information loss with the increasing layers. Extensive experiments on Paris Street View, Places2 and CelebA-HQ datasets demonstrate that our proposed MRM-Net can effectively recover the textures and structures, and performs favorably against other state-of-the-art methods.
dc.language	en
dc.publisher	Association for Computing Machinery (ACM)
dc.relation.ispartof	MM 2022 - Proceedings of the 30th ACM International Conference on Multimedia
dc.relation.ispartof	Proceedings of the 30th ACM International Conference on Multimedia
dc.relation.isbasedon	10.1145/3503161.3548030
dc.rights	info:eu-repo/semantics/closedAccess
dc.title	Deep Multi-Resolution Mutual Learning for Image Inpainting
dc.type	Conference Proceeding
pubs.organisational-group	/University of Technology Sydney
pubs.organisational-group	/University of Technology Sydney/Faculty of Engineering and Information Technology
utslib.copyright.status	closed_access	*
dc.date.updated	2023-05-19T04:49:47Z
pubs.publication-status	Published

Abstract:

Deep image inpainting methods have improved the inpainting performance greatly due to the powerful representation ability of deep learning. However, current deep inpainting networks still tend to produce unreasonable structures and blurry textures due to the ill-posed properties of the task, i.e., image inpainting is still a challenging topic. In this paper, we therefore propose a novel deep multi-resolution mutual learning (DMRML) strategy, which can fully explore the information from various resolutions. Specifically, we design a new image inpainting network, termed multi-resolution mutual network (MRM-Net), which takes the damaged images of different resolutions as input, then excavates and exploits the correlation among different resolutions to guide the image inpainting process. Technically, we designs two new modules called multi-resolution information interaction (MRII) and adaptive content enhancement (ACE). MRII aims at discovering the correlation of multiple resolutions and exchanging information, and ACE focuses on enhancing the contents using the interacted features. Note that we also present an memory preservation mechanism (MPM) to prevent from the information loss with the increasing layers. Extensive experiments on Paris Street View, Places2 and CelebA-HQ datasets demonstrate that our proposed MRM-Net can effectively recover the textures and structures, and performs favorably against other state-of-the-art methods.

Please use this identifier to cite or link to this item:

http://hdl.handle.net/10453/170372