Unsupervised Image-to-Image Translation With Self-Attention and Relativistic Discriminator Adversarial Networks

Lin, H; Ren, S; Yang, Y; Zhang, YY

Unsupervised Image-to-Image Translation With Self-Attention and Relativistic Discriminator Adversarial Networks

Lin, H Ren, S Yang, Y

Zhang, YY

Permalink

Publication Type:: Journal Article
Citation:: Zidonghua Xuebao/Acta Automatica Sinica, 2021, 47, (9), pp. 2226-2237
Issue Date:: 2021-09-01

Closed Access

	Filename	Description	Size
	19110430_7963514620005671.pdf		4.61 MB	Adobe PDF	View/Open

Copyright Clearance Process

Recently Added
In Progress
Closed Access

This item is closed access and not available.

Full metadata record

Field	Value	Language
dc.contributor.author	Lin, H
dc.contributor.author	Ren, S
dc.contributor.author	Yang, Y https://orcid.org/0000-0002-0512-880X
dc.contributor.author	Zhang, YY
dc.date.accessioned	2022-06-17T04:27:59Z
dc.date.available	2022-06-17T04:27:59Z
dc.date.issued	2021-09-01
dc.identifier.citation	Zidonghua Xuebao/Acta Automatica Sinica, 2021, 47, (9), pp. 2226-2237
dc.identifier.issn	0254-4156
dc.identifier.uri	http://hdl.handle.net/10453/158229
dc.description.abstract	Unsupervised image-to-image translation using unpaired training data can accomplish a variety of image translation tasks such as object transformation, seasonal transfer, and satellite and map transformation. The image-to-image translation method based on generative adversarial network (GAN) has not been satisfying due to the following reasons, the training process is unstable and the irrelevant domain changes greatly, the output images are blurred in detail and low in authenticity. This paper proposes an unsupervised image-to-image translation method with self-attention and relativistic discriminator adversarial networks based on dual learning. Firstly, in the generator, self-attention mechanism is designed to build long-short-range dependency for image generation tasks. Skip-connection between low and high convolution layers help reduce the loss of feature information in irrelevant image domain. Then, in the discriminator, spectral normalization is used to prevent the gradient disappearing caused by the mutation of the discrimination ability to enhance training stability. Finally, in the loss function, the self-reconstruction consistency is added on the basis of loop reconstruction to focus on target image domain change. The relativistic adversarial loss is designed to guide the zero-sum game between generator and discriminator. The experimental results from the Horse & Zebra, Summer & Winter, and AerialPhoto & Map datasets demonstrate that compared with the current image translation methods, our method can establish a more realistic image domain mapping relationship and improve the translation quality of the generated image.
dc.language	en
dc.relation.ispartof	Zidonghua Xuebao/Acta Automatica Sinica
dc.relation.isbasedon	10.16383/j.aas.c190074
dc.rights	info:eu-repo/semantics/closedAccess
dc.subject	0102 Applied Mathematics, 0801 Artificial Intelligence and Image Processing, 0906 Electrical and Electronic Engineering
dc.subject.classification	Industrial Engineering & Automation
dc.title	Unsupervised Image-to-Image Translation With Self-Attention and Relativistic Discriminator Adversarial Networks
dc.type	Journal Article
utslib.citation.volume	47
utslib.for	0102 Applied Mathematics
utslib.for	0801 Artificial Intelligence and Image Processing
utslib.for	0906 Electrical and Electronic Engineering
pubs.organisational-group	/University of Technology Sydney
pubs.organisational-group	/University of Technology Sydney/Faculty of Engineering and Information Technology
pubs.organisational-group	/University of Technology Sydney/Strength - AAII - Australian Artificial Intelligence Institute
utslib.copyright.status	closed_access	*
dc.date.updated	2022-06-17T04:27:56Z
pubs.issue	9
pubs.publication-status	Published
pubs.volume	47
utslib.citation.issue	9

Abstract:

Unsupervised image-to-image translation using unpaired training data can accomplish a variety of image translation tasks such as object transformation, seasonal transfer, and satellite and map transformation. The image-to-image translation method based on generative adversarial network (GAN) has not been satisfying due to the following reasons, the training process is unstable and the irrelevant domain changes greatly, the output images are blurred in detail and low in authenticity. This paper proposes an unsupervised image-to-image translation method with self-attention and relativistic discriminator adversarial networks based on dual learning. Firstly, in the generator, self-attention mechanism is designed to build long-short-range dependency for image generation tasks. Skip-connection between low and high convolution layers help reduce the loss of feature information in irrelevant image domain. Then, in the discriminator, spectral normalization is used to prevent the gradient disappearing caused by the mutation of the discrimination ability to enhance training stability. Finally, in the loss function, the self-reconstruction consistency is added on the basis of loop reconstruction to focus on target image domain change. The relativistic adversarial loss is designed to guide the zero-sum game between generator and discriminator. The experimental results from the Horse & Zebra, Summer & Winter, and AerialPhoto & Map datasets demonstrate that compared with the current image translation methods, our method can establish a more realistic image domain mapping relationship and improve the translation quality of the generated image.

Please use this identifier to cite or link to this item:

http://hdl.handle.net/10453/158229