Weakly Supervised Joint Transfer and Regression of Textures for 3-D Human Reconstruction

Zhao, F; Liao, S; Huo, J; Huo, Z; Wang, W; Han, J; Shan, C

Weakly Supervised Joint Transfer and Regression of Textures for 3-D Human Reconstruction

Zhao, F Liao, S Huo, J Huo, Z Wang, W

Han, J Shan, C

Permalink

Publisher:: IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
Publication Type:: Journal Article
Citation:: IEEE Transactions on Consumer Electronics, 2024, 70, (1), pp. 4400-4410
Issue Date:: 2024-02-01

Recently Added

	Filename	Description	Size
	1716013.pdf	Published version	3.89 MB	Adobe PDF	View/Open

Copyright Clearance Process

Recently Added
In Progress
Open Access

This item is new to OPUS and is not currently available.

Full metadata record

Field	Value	Language
dc.contributor.author	Zhao, F
dc.contributor.author	Liao, S
dc.contributor.author	Huo, J
dc.contributor.author	Huo, Z
dc.contributor.author	Wang, W https://orcid.org/0000-0001-8727-1572
dc.contributor.author	Han, J
dc.contributor.author	Shan, C
dc.date.accessioned	2024-08-07T05:05:44Z
dc.date.available	2024-08-07T05:05:44Z
dc.date.issued	2024-02-01
dc.identifier.citation	IEEE Transactions on Consumer Electronics, 2024, 70, (1), pp. 4400-4410
dc.identifier.issn	0098-3063
dc.identifier.issn	1558-4127
dc.identifier.uri	http://hdl.handle.net/10453/180269
dc.description.abstract	3D human model reconstruction from a single image has a wide range of applications in VR/AR, video games, virtual dressing and data synthesis. Texture generation is one of its important components. Existing texture flow based generation methods usually cannot handle well texture inference for invisible human parts in input images, especially for asymmetric texture structures. To address this problem, we propose a joint texture transfer and regression model consisting of part-aware texture transfer, which predicts the texture flow in the UV space from the semantic mesh segmentation, and UV-map based texture regression, which further refines textures for invisible human parts according to the UV texture map produced by the texture transfer. These two modules are learned jointly in a weakly supervised manner based on the perceptual metrics, without using 3D texture supervision. Furthermore, we exploit our generated textures to synthesize human images with various viewpoints, poses and backgrounds as training images for the person re-identification (re-ID) task. Extensive experiments on pedestrian image datasets show that our model produces higher quality textures than other texture generation methods and improves the re-ID performance.
dc.language	English
dc.publisher	IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
dc.relation.ispartof	IEEE Transactions on Consumer Electronics
dc.relation.isbasedon	10.1109/TCE.2024.3373178
dc.rights	info:eu-repo/semantics/restrictedAccess
dc.subject	0906 Electrical and Electronic Engineering, 1005 Communications Technologies
dc.subject.classification	Networking & Telecommunications
dc.subject.classification	4008 Electrical engineering
dc.subject.classification	4009 Electronics, sensors and digital hardware
dc.title	Weakly Supervised Joint Transfer and Regression of Textures for 3-D Human Reconstruction
dc.type	Journal Article
utslib.citation.volume	70
utslib.for	0906 Electrical and Electronic Engineering
utslib.for	1005 Communications Technologies
pubs.organisational-group	University of Technology Sydney
pubs.organisational-group	University of Technology Sydney/Faculty of Engineering and Information Technology
pubs.organisational-group	University of Technology Sydney/Faculty of Engineering and Information Technology/School of Computer Science
utslib.copyright.status	recently_added	*
dc.date.updated	2024-08-07T05:05:38Z
pubs.issue	1
pubs.publication-status	Published
pubs.volume	70
utslib.citation.issue	1

Abstract:

3D human model reconstruction from a single image has a wide range of applications in VR/AR, video games, virtual dressing and data synthesis. Texture generation is one of its important components. Existing texture flow based generation methods usually cannot handle well texture inference for invisible human parts in input images, especially for asymmetric texture structures. To address this problem, we propose a joint texture transfer and regression model consisting of part-aware texture transfer, which predicts the texture flow in the UV space from the semantic mesh segmentation, and UV-map based texture regression, which further refines textures for invisible human parts according to the UV texture map produced by the texture transfer. These two modules are learned jointly in a weakly supervised manner based on the perceptual metrics, without using 3D texture supervision. Furthermore, we exploit our generated textures to synthesize human images with various viewpoints, poses and backgrounds as training images for the person re-identification (re-ID) task. Extensive experiments on pedestrian image datasets show that our model produces higher quality textures than other texture generation methods and improves the re-ID performance.

Please use this identifier to cite or link to this item:

http://hdl.handle.net/10453/180269