MASK-CNN-Transformer for real-time multi-label weather recognition

Chen, S; Shu, T; Zhao, H; Tang, YY

MASK-CNN-Transformer for real-time multi-label weather recognition

Chen, S

Shu, T Zhao, H Tang, YY

Permalink

Publisher:: ELSEVIER
Publication Type:: Journal Article
Citation:: Knowledge-Based Systems, 2023, 278
Issue Date:: 2023-10-25

Closed Access

	Filename	Description	Size
	Mask - CNN transformer for real-time.pdf	Accepted version	3.76 MB	Adobe PDF	View/Open

Copyright Clearance Process

Recently Added
In Progress
Closed Access

This item is closed access and not available.

Full metadata record

Field	Value	Language
dc.contributor.author	Chen, S https://orcid.org/0000-0001-9992-2264
dc.contributor.author	Shu, T
dc.contributor.author	Zhao, H
dc.contributor.author	Tang, YY
dc.date.accessioned	2024-03-25T00:09:28Z
dc.date.available	2024-03-25T00:09:28Z
dc.date.issued	2023-10-25
dc.identifier.citation	Knowledge-Based Systems, 2023, 278
dc.identifier.issn	0950-7051
dc.identifier.issn	1872-7409
dc.identifier.uri	http://hdl.handle.net/10453/177065
dc.description.abstract	Weather recognition is an essential support for many practical life applications, including traffic safety, environment, and meteorology. However, many existing related works cannot comprehensively describe weather conditions due to their complex co-occurrence dependencies. This paper proposes a novel multi-label weather recognition model considering these dependencies. The proposed model called MASK-Convolutional Neural Network-Transformer (MASK-CT) is based on the Transformer, the convolutional process, and the MASK mechanism. The model employs multiple convolutional layers to extract features from weather images and a Transformer encoder to calculate the probability of each weather condition based on the extracted features. To improve the generalization ability of MASK-CT, a MASK mechanism is used during the training phase. The effect of the MASK mechanism is explored and discussed. The Mask mechanism randomly withholds some information from one-pair training instances (one image and its corresponding label). There are two types of MASK methods. Specifically, MASK-I is designed and deployed on the image before feeding it into the weather feature extractor and MASK-II is applied to the image label. The Transformer encoder is then utilized on the randomly masked image features and labels. The experimental results from various real-world weather recognition datasets demonstrate that the proposed MASK-CT model outperforms state-of-the-art methods. Furthermore, the high-speed dynamic real-time weather recognition capability of the MASK-CT is evaluated.
dc.language	English
dc.publisher	ELSEVIER
dc.relation.ispartof	Knowledge-Based Systems
dc.relation.isbasedon	10.1016/j.knosys.2023.110881
dc.rights	info:eu-repo/semantics/closedAccess
dc.subject	08 Information and Computing Sciences, 15 Commerce, Management, Tourism and Services, 17 Psychology and Cognitive Sciences
dc.subject.classification	Artificial Intelligence & Image Processing
dc.subject.classification	4602 Artificial intelligence
dc.subject.classification	4605 Data management and data science
dc.subject.classification	4611 Machine learning
dc.title	MASK-CNN-Transformer for real-time multi-label weather recognition
dc.type	Journal Article
utslib.citation.volume	278
utslib.for	08 Information and Computing Sciences
utslib.for	15 Commerce, Management, Tourism and Services
utslib.for	17 Psychology and Cognitive Sciences
pubs.organisational-group	University of Technology Sydney
pubs.organisational-group	University of Technology Sydney/Faculty of Engineering and Information Technology
pubs.organisational-group	University of Technology Sydney/Faculty of Engineering and Information Technology/School of Computer Science
utslib.copyright.status	closed_access	*
dc.date.updated	2024-03-25T00:09:27Z
pubs.publication-status	Published
pubs.volume	278

Abstract:

Weather recognition is an essential support for many practical life applications, including traffic safety, environment, and meteorology. However, many existing related works cannot comprehensively describe weather conditions due to their complex co-occurrence dependencies. This paper proposes a novel multi-label weather recognition model considering these dependencies. The proposed model called MASK-Convolutional Neural Network-Transformer (MASK-CT) is based on the Transformer, the convolutional process, and the MASK mechanism. The model employs multiple convolutional layers to extract features from weather images and a Transformer encoder to calculate the probability of each weather condition based on the extracted features. To improve the generalization ability of MASK-CT, a MASK mechanism is used during the training phase. The effect of the MASK mechanism is explored and discussed. The Mask mechanism randomly withholds some information from one-pair training instances (one image and its corresponding label). There are two types of MASK methods. Specifically, MASK-I is designed and deployed on the image before feeding it into the weather feature extractor and MASK-II is applied to the image label. The Transformer encoder is then utilized on the randomly masked image features and labels. The experimental results from various real-world weather recognition datasets demonstrate that the proposed MASK-CT model outperforms state-of-the-art methods. Furthermore, the high-speed dynamic real-time weather recognition capability of the MASK-CT is evaluated.

Please use this identifier to cite or link to this item:

http://hdl.handle.net/10453/177065