Towards Realistic Transfer Learning Methods: Theory and Algorithms
- Publication Type:
- Thesis
- Issue Date:
- 2020
Open Access
Copyright Clearance Process
- Recently Added
- In Progress
- Open Access
This item is open access.
Transfer learning aims to leverage knowledge from domains with abundant labels (i.e., source domains) to help train a classifier or predictor for the domain with insufficient labels (i.e., target domain). The trained classifier or predictor is expected to have better performance (e.g., higher accuracy) than classifiers only trained with data in the target domain.
Although recent research of transfer learning has shown a decent ability to transfer knowledge from a source domain to a target domain, most research require certain assumptions to ensure their efficacy. These assumptions are probably not realistic, which means that existing transfer learning methods still face several unsolved and challenging problems in real world.
This thesis aims to address four orthogonal problems faced by existing transfer learning methods: 1) How to test if feature spaces of two domains are from different distributions; 2) How to transfer knowledge when labels in the source domain cannot be perfectly annotated (i.e., the source domain contains noisy labels); 3) How to transfer knowledge when source and target domains have different dimensions (i.e., heterogeneous scenario); and 4) How to transfer knowledge across multiple source domains and a different-dimension target domain.
To address Problem 1), this thesis presents two new two-sample tests to test if the feature spaces of source domains and target domain are from different distributions. One is suitable for low-dimension data (Chapter 3) and another for high-dimension data (Chapter 4). If feature spaces of domains are statistically different, we need to use transfer learning methods on these domains. Moreover, the test statistics used in the proposed tests can be used to measure the distributional discrepancy between two domains.
To address Problem 2), this thesis presents a theoretical bound to show that existing transfer learning methods cannot work well when a source domain contain noisy labels. Then, a novel transfer learning approach is proposed to transfer knowledge across a source domain (with noisy labels) and a target domain. Finally, a generalization bound is proved to explain why the proposed method can reliably transfer knowledge across domains in noisy scenario (Chapter 5).
To address Problem 3), the most challenging problem in the field of domain adaptation, Chapter 6 presents a theorem to show when we can reliably transfer knowledge across two different-dimension (i.e., heterogeneous) domains and propose a solution to this problem. Since methods in Chapter 6 assume that the number of samples in two domains must be the same (i.e., two balanced domains), Chapter 7 presents a novel fuzzy-relation based method to transfer knowledge across two imbalanced domains.
To address Problem 4), Chapter 8 presents a novel fuzzy-relation neural network to transfer knowledge from multiple source domains to a target domain, where any of two domains are heterogeneous (i.e., feature spaces of any of two domains have different dimensions).
To conclude, this thesis not only propose a set of effective methods for realistic transfer learning, but also contribute to theory of transfer learning.
Please use this identifier to cite or link to this item: