RNNtcs: A test case selection method for Recurrent Neural Networks

Wu, X; Shen, J; Zheng, W; Lin, L; Sui, Y; Semasaba, AOA

RNNtcs: A test case selection method for Recurrent Neural Networks

Wu, X Shen, J Zheng, W Lin, L Sui, Y

Semasaba, AOA

Permalink

Publisher:: ELSEVIER
Publication Type:: Journal Article
Citation:: Knowledge-Based Systems, 2023, 279
Issue Date:: 2023-11-04

Closed Access

	Filename	Description	Size
	RNNtcs A test case selection method for Recurrent Neural Networks_Published.pdf	Published version	1.74 MB	Adobe PDF	View/Open

Copyright Clearance Process

Recently Added
In Progress
Closed Access

This item is closed access and not available.

Full metadata record

Field	Value	Language
dc.contributor.author	Wu, X
dc.contributor.author	Shen, J
dc.contributor.author	Zheng, W
dc.contributor.author	Lin, L
dc.contributor.author	Sui, Y https://orcid.org/0000-0002-9510-6574
dc.contributor.author	Semasaba, AOA
dc.date.accessioned	2024-02-22T05:04:19Z
dc.date.available	2024-02-22T05:04:19Z
dc.date.issued	2023-11-04
dc.identifier.citation	Knowledge-Based Systems, 2023, 279
dc.identifier.issn	0950-7051
dc.identifier.issn	1872-7409
dc.identifier.uri	http://hdl.handle.net/10453/175822
dc.description.abstract	Recurrent Neural Network (RNN) is a typical feedback neural network, which is particularly effective in processing time-series data tasks such as image description, text generation and classification, etc. However, its safety and quality issues have aroused strong concern in academia and industry. One of the most commonly used methods to ensure RNN-driven software quality is testing and optimizing RNN. Its core idea is to introduce human efforts to label test cases so that many labeled test cases can be used to trigger faults in RNNs. But this brings a huge overhead. Therefore, it is of great importance to study the test case selection method for RNN. Current research on test case selection for RNN has achieved some results from the stateful view of RNN, but there are still some problems, such as a low bug detection rate, poor inclusiveness, and lack of diversity. In this paper, according to the structural characteristics of RNN, we propose a method of RNN test case selection, RNNtcs, which combines clustering and uncertainty, in order to select test cases that can touch RNN bugs as much as possible, thus reducing the cost of labeling. In order to evaluate RNNtcs, we conduct experiments on popular image datasets, text datasets and RNN models. The experimental results show that RNNtcs is superior to the state-of-the-art approach DeepState in bug detection effectiveness, bug detection inclusiveness and bug detection diversity. The test cases selected by RNNtcs can effectively improve the robustness of the RNN model.
dc.language	English
dc.publisher	ELSEVIER
dc.relation.ispartof	Knowledge-Based Systems
dc.relation.isbasedon	10.1016/j.knosys.2023.110955
dc.rights	info:eu-repo/semantics/closedAccess
dc.subject	08 Information and Computing Sciences, 15 Commerce, Management, Tourism and Services, 17 Psychology and Cognitive Sciences
dc.subject.classification	Artificial Intelligence & Image Processing
dc.subject.classification	4602 Artificial intelligence
dc.subject.classification	4605 Data management and data science
dc.subject.classification	4611 Machine learning
dc.title	RNNtcs: A test case selection method for Recurrent Neural Networks
dc.type	Journal Article
utslib.citation.volume	279
utslib.for	08 Information and Computing Sciences
utslib.for	15 Commerce, Management, Tourism and Services
utslib.for	17 Psychology and Cognitive Sciences
pubs.organisational-group	University of Technology Sydney
pubs.organisational-group	University of Technology Sydney/Faculty of Engineering and Information Technology
pubs.organisational-group	University of Technology Sydney/Strength - AAII - Australian Artificial Intelligence Institute
utslib.copyright.status	closed_access	*
dc.date.updated	2024-02-22T05:04:15Z
pubs.publication-status	Published
pubs.volume	279

Abstract:

Recurrent Neural Network (RNN) is a typical feedback neural network, which is particularly effective in processing time-series data tasks such as image description, text generation and classification, etc. However, its safety and quality issues have aroused strong concern in academia and industry. One of the most commonly used methods to ensure RNN-driven software quality is testing and optimizing RNN. Its core idea is to introduce human efforts to label test cases so that many labeled test cases can be used to trigger faults in RNNs. But this brings a huge overhead. Therefore, it is of great importance to study the test case selection method for RNN. Current research on test case selection for RNN has achieved some results from the stateful view of RNN, but there are still some problems, such as a low bug detection rate, poor inclusiveness, and lack of diversity. In this paper, according to the structural characteristics of RNN, we propose a method of RNN test case selection, RNNtcs, which combines clustering and uncertainty, in order to select test cases that can touch RNN bugs as much as possible, thus reducing the cost of labeling. In order to evaluate RNNtcs, we conduct experiments on popular image datasets, text datasets and RNN models. The experimental results show that RNNtcs is superior to the state-of-the-art approach DeepState in bug detection effectiveness, bug detection inclusiveness and bug detection diversity. The test cases selected by RNNtcs can effectively improve the robustness of the RNN model.

Please use this identifier to cite or link to this item:

http://hdl.handle.net/10453/175822