Is Out-of-Distribution Detection Learnable?

Field	Value	Language
dc.contributor.author	Fang, Z https://orcid.org/0000-0003-0602-6255
dc.contributor.author	Li, Y
dc.contributor.author	Lu, J
dc.contributor.author	Dong, J
dc.contributor.author	Han, B
dc.contributor.author	Liu, F https://orcid.org/0000-0002-5005-9129
dc.date	2022-11-26
dc.date.accessioned	2024-02-28T12:26:35Z
dc.date.available	2024-02-28T12:26:35Z
dc.date.issued	2023-01-01
dc.identifier.citation	Advances in Neural Information Processing Systems, 2023, 35
dc.identifier.isbn	9781713871088
dc.identifier.issn	1049-5258
dc.identifier.uri	http://hdl.handle.net/10453/175945
dc.description.abstract	Supervised learning aims to train a classifier under the assumption that training and test data are from the same distribution. To ease the above assumption, researchers have studied a more realistic setting: out-of-distribution (OOD) detection, where test data may come from classes that are unknown during training (i.e., OOD data). Due to the unavailability and diversity of OOD data, good generalization ability is crucial for effective OOD detection algorithms. To study the generalization of OOD detection, in this paper, we investigate the probably approximately correct (PAC) learning theory of OOD detection, which is proposed by researchers as an open problem. First, we find a necessary condition for the learnability of OOD detection. Then, using this condition, we prove several impossibility theorems for the learnability of OOD detection under some scenarios. Although the impossibility theorems are frustrating, we find that some conditions of these impossibility theorems may not hold in some practical scenarios. Based on this observation, we next give several necessary and sufficient conditions to characterize the learnability of OOD detection in some practical scenarios. Lastly, we also offer theoretical supports for several representative OOD detection works based on our OOD theory.
dc.language	en
dc.relation.ispartof	Advances in Neural Information Processing Systems
dc.relation.ispartof	Advances in Neural Information Processing Systems (NeurIPS 2022),
dc.rights	info:eu-repo/semantics/openAccess
dc.subject	1701 Psychology, 1702 Cognitive Sciences
dc.subject.classification	4611 Machine learning
dc.title	Is Out-of-Distribution Detection Learnable?
dc.type	Conference Proceeding
utslib.citation.volume	35
utslib.location.activity	New Orleans
utslib.for	1701 Psychology
utslib.for	1702 Cognitive Sciences
pubs.organisational-group	University of Technology Sydney
pubs.organisational-group	University of Technology Sydney/Faculty of Engineering and Information Technology
pubs.organisational-group	University of Technology Sydney/Strength - AAII - Australian Artificial Intelligence Institute
utslib.copyright.status	open_access	*
pubs.consider-herdc	false
dc.date.updated	2024-02-28T12:26:33Z
pubs.finish-date	2022-12-09
pubs.place-of-publication	Advances in Neural Information Processing Systems (NeurIPS 2022),
pubs.publication-status	Published online
pubs.start-date	2022-11-26
pubs.volume	35
dc.location	Advances in Neural Information Processing Systems (NeurIPS 2022),

Abstract:

Supervised learning aims to train a classifier under the assumption that training and test data are from the same distribution. To ease the above assumption, researchers have studied a more realistic setting: out-of-distribution (OOD) detection, where test data may come from classes that are unknown during training (i.e., OOD data). Due to the unavailability and diversity of OOD data, good generalization ability is crucial for effective OOD detection algorithms. To study the generalization of OOD detection, in this paper, we investigate the probably approximately correct (PAC) learning theory of OOD detection, which is proposed by researchers as an open problem. First, we find a necessary condition for the learnability of OOD detection. Then, using this condition, we prove several impossibility theorems for the learnability of OOD detection under some scenarios. Although the impossibility theorems are frustrating, we find that some conditions of these impossibility theorems may not hold in some practical scenarios. Based on this observation, we next give several necessary and sufficient conditions to characterize the learnability of OOD detection in some practical scenarios. Lastly, we also offer theoretical supports for several representative OOD detection works based on our OOD theory.

Please use this identifier to cite or link to this item:

http://hdl.handle.net/10453/175945