Where and How to Transfer: Knowledge Aggregation-Induced Transferability Perception for Unsupervised Domain Adaptation

Jiahua, D; Yang, C; Fang, Z; Zhengming, D

Where and How to Transfer: Knowledge Aggregation-Induced Transferability Perception for Unsupervised Domain Adaptation

Jiahua, D Yang, C Fang, Z

Zhengming, D

Permalink

Publisher:: Institute of Electrical and Electronics Engineers
Publication Type:: Journal Article
Citation:: IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 14, (8), pp. 1-17
Issue Date:: 2022-08-07

Closed Access

	Filename	Description	Size
	Where_and_How_to_Transfer_Knowledge_Aggregation-Induced_Transferability_Perception_for_Unsupervised_Domain_Adaptation.pdf	Published version	11.51 MB	Adobe PDF	View/Open

Copyright Clearance Process

Recently Added
In Progress
Closed Access

This item is closed access and not available.

Full metadata record

Field	Value	Language
dc.contributor.author	Jiahua, D
dc.contributor.author	Yang, C
dc.contributor.author	Fang, Z https://orcid.org/0000-0003-0602-6255
dc.contributor.author	Zhengming, D
dc.date.accessioned	2023-01-23T03:52:07Z
dc.date.available	2023-01-23T03:52:07Z
dc.date.issued	2022-08-07
dc.identifier.citation	IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 14, (8), pp. 1-17
dc.identifier.issn	0162-8828
dc.identifier.issn	1939-3539
dc.identifier.uri	http://hdl.handle.net/10453/165361
dc.description.abstract	Unsupervised domain adaptation without accessing expensive annotation processes of target data has achieved remarkable successes in semantic segmentation. However, most existing state-of-the-art methods cannot explore whether semantic representations across domains are transferable or not, which may result in the negative transfer brought by irrelevant knowledge. To tackle this challenge, in this paper, we develop a novel Knowledge Aggregation-induced Transferability Perception (KATP) module for unsupervised domain adaptation, which is a pioneering attempt to distinguish transferable or untransferable knowledge across domains. Specifically, the KATP module is designed to quantify which semantic knowledge across domains is transferable, by incorporating the transferability information propagation from constructed global category-wise prototypes. Based on KATP, we design a novel KATP Adaptation Network (KATPAN) to determine where and how to transfer. The KATPAN contains a transferable appearance translation module TA(·) and a transferable representation augmentation module TR(·), where both modules construct a virtuous circle of performance promotion. TA(·) develops a transferability-aware information bottleneck to highlight where to adapt transferable visual characterizations and modality information; TR(·) explores how to augment transferable representations while abandoning untransferable information, and promotes the translation performance of TA(·) in return. Comprehensive experiments on several representative benchmark datasets and a medical dataset support the state-of-the-art performance of our model.
dc.format	Print-Electronic
dc.language	eng
dc.publisher	Institute of Electrical and Electronics Engineers
dc.relation.ispartof	IEEE Transactions on Pattern Analysis and Machine Intelligence
dc.relation.isbasedon	10.1109/TPAMI.2021.3128560
dc.rights	info:eu-repo/semantics/closedAccess
dc.subject	0801 Artificial Intelligence and Image Processing, 0806 Information Systems, 0906 Electrical and Electronic Engineering
dc.subject.classification	Artificial Intelligence & Image Processing
dc.title	Where and How to Transfer: Knowledge Aggregation-Induced Transferability Perception for Unsupervised Domain Adaptation
dc.type	Journal Article
utslib.citation.volume	14
utslib.location.activity	United States
utslib.for	0801 Artificial Intelligence and Image Processing
utslib.for	0806 Information Systems
utslib.for	0906 Electrical and Electronic Engineering
pubs.organisational-group	/University of Technology Sydney
pubs.organisational-group	/University of Technology Sydney/Faculty of Engineering and Information Technology
pubs.organisational-group	/University of Technology Sydney/Strength - AAII - Australian Artificial Intelligence Institute
utslib.copyright.status	closed_access	*
pubs.consider-herdc	false
dc.date.updated	2023-01-23T03:52:00Z
pubs.issue	8
pubs.publication-status	Submitted
pubs.volume	14
utslib.citation.issue	8

Abstract:

Unsupervised domain adaptation without accessing expensive annotation processes of target data has achieved remarkable successes in semantic segmentation. However, most existing state-of-the-art methods cannot explore whether semantic representations across domains are transferable or not, which may result in the negative transfer brought by irrelevant knowledge. To tackle this challenge, in this paper, we develop a novel Knowledge Aggregation-induced Transferability Perception (KATP) module for unsupervised domain adaptation, which is a pioneering attempt to distinguish transferable or untransferable knowledge across domains. Specifically, the KATP module is designed to quantify which semantic knowledge across domains is transferable, by incorporating the transferability information propagation from constructed global category-wise prototypes. Based on KATP, we design a novel KATP Adaptation Network (KATPAN) to determine where and how to transfer. The KATPAN contains a transferable appearance translation module TA(·) and a transferable representation augmentation module TR(·), where both modules construct a virtuous circle of performance promotion. TA(·) develops a transferability-aware information bottleneck to highlight where to adapt transferable visual characterizations and modality information; TR(·) explores how to augment transferable representations while abandoning untransferable information, and promotes the translation performance of TA(·) in return. Comprehensive experiments on several representative benchmark datasets and a medical dataset support the state-of-the-art performance of our model.

Please use this identifier to cite or link to this item:

http://hdl.handle.net/10453/165361