Adaptive tree-like neural network: Overcoming catastrophic forgetting to classify streaming data with concept drifts

Wen, YM; Liu, X; Yu, H

Adaptive tree-like neural network: Overcoming catastrophic forgetting to classify streaming data with concept drifts

Wen, YM Liu, X Yu, H

Permalink

Publisher:: ELSEVIER
Publication Type:: Journal Article
Citation:: Knowledge-Based Systems, 2024, 293
Issue Date:: 2024-06-07

Closed Access

	Filename	Description	Size
	1-s2.0-S0950705124002715-main.pdf	Published version	3.3 MB		View/Open

Copyright Clearance Process

Recently Added
In Progress
Closed Access

This item is closed access and not available.

Full metadata record

Field	Value	Language
dc.contributor.author	Wen, YM
dc.contributor.author	Liu, X
dc.contributor.author	Yu, H https://orcid.org/0000-0003-3444-9992
dc.date.accessioned	2025-02-24T03:08:40Z
dc.date.available	2025-02-24T03:08:40Z
dc.date.issued	2024-06-07
dc.identifier.citation	Knowledge-Based Systems, 2024, 293
dc.identifier.issn	0950-7051
dc.identifier.issn	1872-7409
dc.identifier.uri	http://hdl.handle.net/10453/185280
dc.description.abstract	With the development of deep neural networks (DNNs), classifying streaming data with concept drifts based on DNNs is becoming more and more effective. However, the continuous and infinite characteristic of streaming data makes it difficult to set an appropriate depth for a DNN-based model in advance. Moreover, how to improve the model's adaptability to concept drifts while overcoming catastrophic forgetting still remains a difficult issue. To address these issues, an Adaptive Tree-like Neural Network (ATNN) is proposed in this paper. ATNN adaptively increases the depth of its active branch according to the weight of the deepest node in the active branch. Once a new concept is detected, it chooses a suitable position on its trunk to grow a branch for the new concept according to the relation between the Fisher information and the gradient of parameters. Experiments demonstrate the rationality of ATNN to adaptively increase the depth of its branch or to choose a suitable position to grow a new branch, and illustrate that ATNN can more quickly adapt to concept drifts, and continuously improve classification performance for recurring concepts. The code of the proposed algorithm is available at https://github.com/mlmmwym/ATNN.
dc.language	English
dc.publisher	ELSEVIER
dc.relation.ispartof	Knowledge-Based Systems
dc.relation.isbasedon	10.1016/j.knosys.2024.111636
dc.rights	info:eu-repo/semantics/closedAccess
dc.subject	08 Information and Computing Sciences, 15 Commerce, Management, Tourism and Services, 17 Psychology and Cognitive Sciences
dc.subject.classification	Artificial Intelligence & Image Processing
dc.subject.classification	4602 Artificial intelligence
dc.subject.classification	4605 Data management and data science
dc.subject.classification	4611 Machine learning
dc.title	Adaptive tree-like neural network: Overcoming catastrophic forgetting to classify streaming data with concept drifts
dc.type	Journal Article
utslib.citation.volume	293
utslib.for	08 Information and Computing Sciences
utslib.for	15 Commerce, Management, Tourism and Services
utslib.for	17 Psychology and Cognitive Sciences
pubs.organisational-group	University of Technology Sydney
pubs.organisational-group	University of Technology Sydney/Faculty of Engineering and Information Technology
utslib.copyright.status	closed_access	*
dc.date.updated	2025-02-24T03:08:38Z
pubs.publication-status	Published
pubs.volume	293

Abstract:

With the development of deep neural networks (DNNs), classifying streaming data with concept drifts based on DNNs is becoming more and more effective. However, the continuous and infinite characteristic of streaming data makes it difficult to set an appropriate depth for a DNN-based model in advance. Moreover, how to improve the model's adaptability to concept drifts while overcoming catastrophic forgetting still remains a difficult issue. To address these issues, an Adaptive Tree-like Neural Network (ATNN) is proposed in this paper. ATNN adaptively increases the depth of its active branch according to the weight of the deepest node in the active branch. Once a new concept is detected, it chooses a suitable position on its trunk to grow a branch for the new concept according to the relation between the Fisher information and the gradient of parameters. Experiments demonstrate the rationality of ATNN to adaptively increase the depth of its branch or to choose a suitable position to grow a new branch, and illustrate that ATNN can more quickly adapt to concept drifts, and continuously improve classification performance for recurring concepts. The code of the proposed algorithm is available at https://github.com/mlmmwym/ATNN.

Please use this identifier to cite or link to this item:

http://hdl.handle.net/10453/185280