Copy and Paste GAN: Face Hallucination from Shaded Thumbnails

Zhang, Y; Tsang, IW; Luo, Y; Hu, CH; Lu, X; Yu, X

Copy and Paste GAN: Face Hallucination from Shaded Thumbnails

Zhang, Y Tsang, IW Luo, Y Hu, CH Lu, X Yu, X

Permalink

Publisher:: IEEE
Publication Type:: Conference Proceeding
Citation:: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2020, 00, pp. 7353-7362
Issue Date:: 2020-01-01

Closed Access

	Filename	Description	Size
	09156345.pdf	Published version	757.18 kB	Adobe PDF	View/Open

Copyright Clearance Process

Recently Added
In Progress
Closed Access

This item is closed access and not available.

Full metadata record

Field	Value	Language
dc.contributor.author	Zhang, Y
dc.contributor.author	Tsang, IW
dc.contributor.author	Luo, Y
dc.contributor.author	Hu, CH
dc.contributor.author	Lu, X
dc.contributor.author	Yu, X https://orcid.org/0000-0002-0269-5649
dc.date	2020-06-13
dc.date.accessioned	2021-04-15T10:33:22Z
dc.date.available	2021-04-15T10:33:22Z
dc.date.issued	2020-01-01
dc.identifier.citation	Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2020, 00, pp. 7353-7362
dc.identifier.isbn	978-1-7281-7168-5
dc.identifier.issn	1063-6919
dc.identifier.issn	2575-7075
dc.identifier.uri	http://hdl.handle.net/10453/148145
dc.description.abstract	Existing face hallucination methods based on convolutional neural networks (CNN) have achieved impressive performance on low-resolution (LR) faces in a normal illumination condition. However, their performance degrades dramatically when LR faces are captured in low or non-uniform illumination conditions. This paper proposes a Copy and Paste Generative Adversarial Network (CPGAN) to recover authentic high-resolution (HR) face images while compensating for low and non-uniform illumination. To this end, we develop two key components in our CPGAN: internal and external Copy and Paste nets (CPnets). Specifically, our internal CPnet exploits facial information residing in the input image to enhance facial details; while our external CPnet leverages an external HR face for illumination compensation. A new illumination compensation loss is thus developed to capture illumination from the external guided face image effectively. Furthermore, our method offsets illumination and upsamples facial details alternatively in a coarse-to-fine fashion, thus alleviating the correspondence ambiguity between LR inputs and external HR inputs. Extensive experiments demonstrate that our method manifests authentic HR face images in a uniform illumination condition and outperforms state-of-the-art methods qualitatively and quantitatively.
dc.language	en
dc.publisher	IEEE
dc.relation	http://purl.org/au-research/grants/arc/DP180100106
dc.relation	http://purl.org/au-research/grants/arc/DP200101328
dc.relation.ispartof	Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition
dc.relation.ispartof	IEEE/CVF Conference on Computer Vision and Pattern Recognition
dc.relation.isbasedon	10.1109/CVPR42600.2020.00738
dc.rights	info:eu-repo/semantics/closedAccess
dc.title	Copy and Paste GAN: Face Hallucination from Shaded Thumbnails
dc.type	Conference Proceeding
utslib.citation.volume	00
utslib.location.activity	Seattle, WA, USA
utslib.for	0801 Artificial Intelligence and Image Processing
pubs.organisational-group	/University of Technology Sydney
pubs.organisational-group	/University of Technology Sydney/Faculty of Engineering and Information Technology
pubs.organisational-group	/University of Technology Sydney/Strength - AAII - Australian Artificial Intelligence Institute
pubs.organisational-group	/University of Technology Sydney/Faculty of Engineering and Information Technology/School of Computer Science
utslib.copyright.status	closed_access	*
pubs.consider-herdc	false
dc.date.updated	2021-04-15T10:33:21Z
pubs.finish-date	2020-06-19
pubs.place-of-publication	Piscataway, USA
pubs.publication-status	Published
pubs.start-date	2020-06-13
pubs.volume	00
dc.location	Piscataway, USA

Abstract:

Existing face hallucination methods based on convolutional neural networks (CNN) have achieved impressive performance on low-resolution (LR) faces in a normal illumination condition. However, their performance degrades dramatically when LR faces are captured in low or non-uniform illumination conditions. This paper proposes a Copy and Paste Generative Adversarial Network (CPGAN) to recover authentic high-resolution (HR) face images while compensating for low and non-uniform illumination. To this end, we develop two key components in our CPGAN: internal and external Copy and Paste nets (CPnets). Specifically, our internal CPnet exploits facial information residing in the input image to enhance facial details; while our external CPnet leverages an external HR face for illumination compensation. A new illumination compensation loss is thus developed to capture illumination from the external guided face image effectively. Furthermore, our method offsets illumination and upsamples facial details alternatively in a coarse-to-fine fashion, thus alleviating the correspondence ambiguity between LR inputs and external HR inputs. Extensive experiments demonstrate that our method manifests authentic HR face images in a uniform illumination condition and outperforms state-of-the-art methods qualitatively and quantitatively.

Please use this identifier to cite or link to this item:

http://hdl.handle.net/10453/148145