How doppelgänger effects in biomedical data confound machine learning.
- Publisher:
- Elsevier
- Publication Type:
- Journal Article
- Citation:
- Drug Discov Today, 2022, 27, (3), pp. 678-685
- Issue Date:
- 2022-03
Closed Access
| Filename | Description | Size | |||
|---|---|---|---|---|---|
| How doppelgänger effects in biomedical data confound machine learning..pdf | Published version | 1.39 MB |
Copyright Clearance Process
- Recently Added
- In Progress
- Closed Access
This item is closed access and not available.
Machine learning (ML) models have been increasingly adopted in drug development for faster identification of potential targets. Cross-validation techniques are commonly used to evaluate these models. However, the reliability of such validation methods can be affected by the presence of data doppelgängers. Data doppelgängers occur when independently derived data are very similar to each other, causing models to perform well regardless of how they are trained (i.e., the doppelgänger effect). Despite the abundance of data doppelgängers in biomedical data and their inflationary effects, they remain uncharacterized. We show their prevalence in biomedical data, demonstrate how doppelgängers arise, and provide proof of their confounding effects. To mitigate the doppelgänger effect, we recommend identifying data doppelgängers before the training-validation split.
Please use this identifier to cite or link to this item: