A Comparative Study of Class Rebalancing Methods for Security Bug Report Classification

Zheng, W; Xun, Y; Wu, X; Deng, Z; Chen, X; Sui, Y

A Comparative Study of Class Rebalancing Methods for Security Bug Report Classification

Zheng, W Xun, Y Wu, X Deng, Z Chen, X Sui, Y

Permalink

Publisher:: Institute of Electrical and Electronics Engineers (IEEE)
Publication Type:: Journal Article
Citation:: IEEE Transactions on Reliability, 2021, 70, (4), pp. 1658-1670
Issue Date:: 2021-12-01

Closed Access

	Filename	Description	Size
	A_Comparative_Study_of_Class_Rebalancing_Methods_for_Security_Bug_Report_Classification.pdf	Published version	1.5 MB	Adobe PDF	View/Open

Copyright Clearance Process

Recently Added
In Progress
Closed Access

This item is closed access and not available.

Full metadata record

Field	Value	Language
dc.contributor.author	Zheng, W
dc.contributor.author	Xun, Y
dc.contributor.author	Wu, X
dc.contributor.author	Deng, Z
dc.contributor.author	Chen, X
dc.contributor.author	Sui, Y https://orcid.org/0000-0002-9510-6574
dc.date.accessioned	2022-05-20T14:05:38Z
dc.date.available	2022-05-20T14:05:38Z
dc.date.issued	2021-12-01
dc.identifier.citation	IEEE Transactions on Reliability, 2021, 70, (4), pp. 1658-1670
dc.identifier.issn	0018-9529
dc.identifier.issn	1558-1721
dc.identifier.uri	http://hdl.handle.net/10453/157574
dc.description.abstract	Identifying security bug reports (SBRs) accurately from a bug repository can reduce a software product's security risk. However, the class imbalance problem exists for SBR prediction since the number of SBRs is often limited, and this issue has not been thoroughly investigated in previous studies. In our study, we choose six real-world projects of different sizes with over 120 000 bug reports in total as our empirical subjects. We first analyze the impact of the class imbalance issue on SBR prediction and confirm its negative impact on prediction performance. Then we perform a comparative study of six state-of-the-art class rebalancing methods combined with five popular classification algorithms for SBR prediction. By comparing with the baseline method Farsec, using the class rebalancing methods can improve the performance in 78% of cases in the worst case. Moreover, the combination of the Rose and random forest classification algorithm can construct the model with the best performance, which increases the performance by 267% in the best case and 75% on average in terms of F1-score. Finally, we summarize eight main findings based on our empirical studies' results, which can provide guidelines for choosing appropriate class rebalancing methods and classifiers for SBR prediction in practice.
dc.language	en
dc.publisher	Institute of Electrical and Electronics Engineers (IEEE)
dc.relation.ispartof	IEEE Transactions on Reliability
dc.relation.isbasedon	10.1109/TR.2021.3118026
dc.rights	info:eu-repo/semantics/closedAccess
dc.subject	0803 Computer Software, 0906 Electrical and Electronic Engineering
dc.subject.classification	Operations Research
dc.title	A Comparative Study of Class Rebalancing Methods for Security Bug Report Classification
dc.type	Journal Article
utslib.citation.volume	70
utslib.for	0803 Computer Software
utslib.for	0906 Electrical and Electronic Engineering
pubs.organisational-group	/University of Technology Sydney
pubs.organisational-group	/University of Technology Sydney/Faculty of Engineering and Information Technology
pubs.organisational-group	/University of Technology Sydney/Strength - AAII - Australian Artificial Intelligence Institute
pubs.organisational-group	/University of Technology Sydney/Faculty of Engineering and Information Technology/School of Computer Science
utslib.copyright.status	closed_access	*
dc.date.updated	2022-05-20T14:05:37Z
pubs.issue	4
pubs.publication-status	Published
pubs.volume	70
utslib.citation.issue	4

Abstract:

Identifying security bug reports (SBRs) accurately from a bug repository can reduce a software product's security risk. However, the class imbalance problem exists for SBR prediction since the number of SBRs is often limited, and this issue has not been thoroughly investigated in previous studies. In our study, we choose six real-world projects of different sizes with over 120 000 bug reports in total as our empirical subjects. We first analyze the impact of the class imbalance issue on SBR prediction and confirm its negative impact on prediction performance. Then we perform a comparative study of six state-of-the-art class rebalancing methods combined with five popular classification algorithms for SBR prediction. By comparing with the baseline method Farsec, using the class rebalancing methods can improve the performance in 78% of cases in the worst case. Moreover, the combination of the Rose and random forest classification algorithm can construct the model with the best performance, which increases the performance by 267% in the best case and 75% on average in terms of F1-score. Finally, we summarize eight main findings based on our empirical studies' results, which can provide guidelines for choosing appropriate class rebalancing methods and classifiers for SBR prediction in practice.

Please use this identifier to cite or link to this item:

http://hdl.handle.net/10453/157574