An octonion-based nonlinear echo state network for speech emotion recognition in Metaverse.

Daneshfar, F; Jamshidi, MB

An octonion-based nonlinear echo state network for speech emotion recognition in Metaverse.

Daneshfar, F Jamshidi, MB

Permalink

Publisher:: PERGAMON-ELSEVIER SCIENCE LTD
Publication Type:: Journal Article
Citation:: Neural Netw, 2023, 163, pp. 108-121
Issue Date:: 2023-06

Closed Access

	Filename	Description	Size
	1-s2.0-S0893608023001600-main.pdf	Published version	3.53 MB	Adobe PDF	View/Open

Copyright Clearance Process

Recently Added
In Progress
Closed Access

This item is closed access and not available.

Full metadata record

Field	Value	Language
dc.contributor.author	Daneshfar, F
dc.contributor.author	Jamshidi, MB
dc.date.accessioned	2024-05-17T00:33:20Z
dc.date.available	2023-03-19
dc.date.available	2024-05-17T00:33:20Z
dc.date.issued	2023-06
dc.identifier.citation	Neural Netw, 2023, 163, pp. 108-121
dc.identifier.issn	0893-6080
dc.identifier.issn	1879-2782
dc.identifier.uri	http://hdl.handle.net/10453/179007
dc.description.abstract	While the Metaverse is becoming a popular trend and drawing much attention from academia, society, and businesses, processing cores used in its infrastructures need to be improved, particularly in terms of signal processing and pattern recognition. Accordingly, the speech emotion recognition (SER) method plays a crucial role in creating the Metaverse platforms more usable and enjoyable for its users. However, existing SER methods continue to be plagued by two significant problems in the online environment. The shortage of adequate engagement and customization between avatars and users is recognized as the first issue and the second problem is related to the complexity of SER problems in the Metaverse as we face people and their digital twins or avatars. This is why developing efficient machine learning (ML) techniques specified for hypercomplex signal processing is essential to enhance the impressiveness and tangibility of the Metaverse platforms. As a solution, echo state networks (ESNs), which are an ML powerful tool for SER, can be an appropriate technique to enhance the Metaverse's foundations in this area. Nevertheless, ESNs have some technical issues restricting them from a precise and reliable analysis, especially in the aspect of high-dimensional data. The most significant limitation of these networks is the high memory consumption caused by their reservoir structure in face of high-dimensional signals. To solve all problems associated with ESNs and their application in the Metaverse, we have come up with a novel structure for ESNs empowered by octonion algebra called NO2GESNet. Octonion numbers have eight dimensions, compactly display high-dimensional data, and improve the network precision and performance in comparison to conventional ESNs. The proposed network also solves the weaknesses of the ESNs in the presentation of the higher-order statistics to the output layer by equipping it with a multidimensional bilinear filter. Three comprehensive scenarios to use the proposed network in the Metaverse have been designed and analyzed, not only do they show the accuracy and performance of the proposed approach, but also the ways how SER can be employed in the Metaverse platforms.
dc.format	Print-Electronic
dc.language	eng
dc.publisher	PERGAMON-ELSEVIER SCIENCE LTD
dc.relation.ispartof	Neural Netw
dc.relation.isbasedon	10.1016/j.neunet.2023.03.026
dc.rights	info:eu-repo/semantics/closedAccess
dc.subject.classification	Artificial Intelligence & Image Processing
dc.subject.classification	4602 Artificial intelligence
dc.subject.classification	4611 Machine learning
dc.subject.classification	4905 Statistics
dc.subject.mesh	Humans
dc.subject.mesh	Neural Networks, Computer
dc.subject.mesh	Speech
dc.subject.mesh	Emotions
dc.subject.mesh	Time
dc.subject.mesh	Machine Learning
dc.subject.mesh	Humans
dc.subject.mesh	Speech
dc.subject.mesh	Emotions
dc.subject.mesh	Time
dc.subject.mesh	Machine Learning
dc.subject.mesh	Neural Networks, Computer
dc.subject.mesh	Humans
dc.subject.mesh	Neural Networks, Computer
dc.subject.mesh	Speech
dc.subject.mesh	Emotions
dc.subject.mesh	Time
dc.subject.mesh	Machine Learning
dc.title	An octonion-based nonlinear echo state network for speech emotion recognition in Metaverse.
dc.type	Journal Article
utslib.citation.volume	163
utslib.location.activity	United States
pubs.organisational-group	University of Technology Sydney
pubs.organisational-group	University of Technology Sydney/Faculty of Engineering and Information Technology
pubs.organisational-group	University of Technology Sydney/Faculty of Engineering and Information Technology/School of Electrical and Data Engineering
utslib.copyright.status	closed_access	*
dc.date.updated	2024-05-17T00:33:18Z
pubs.publication-status	Published
pubs.volume	163

Abstract:

While the Metaverse is becoming a popular trend and drawing much attention from academia, society, and businesses, processing cores used in its infrastructures need to be improved, particularly in terms of signal processing and pattern recognition. Accordingly, the speech emotion recognition (SER) method plays a crucial role in creating the Metaverse platforms more usable and enjoyable for its users. However, existing SER methods continue to be plagued by two significant problems in the online environment. The shortage of adequate engagement and customization between avatars and users is recognized as the first issue and the second problem is related to the complexity of SER problems in the Metaverse as we face people and their digital twins or avatars. This is why developing efficient machine learning (ML) techniques specified for hypercomplex signal processing is essential to enhance the impressiveness and tangibility of the Metaverse platforms. As a solution, echo state networks (ESNs), which are an ML powerful tool for SER, can be an appropriate technique to enhance the Metaverse's foundations in this area. Nevertheless, ESNs have some technical issues restricting them from a precise and reliable analysis, especially in the aspect of high-dimensional data. The most significant limitation of these networks is the high memory consumption caused by their reservoir structure in face of high-dimensional signals. To solve all problems associated with ESNs and their application in the Metaverse, we have come up with a novel structure for ESNs empowered by octonion algebra called NO2GESNet. Octonion numbers have eight dimensions, compactly display high-dimensional data, and improve the network precision and performance in comparison to conventional ESNs. The proposed network also solves the weaknesses of the ESNs in the presentation of the higher-order statistics to the output layer by equipping it with a multidimensional bilinear filter. Three comprehensive scenarios to use the proposed network in the Metaverse have been designed and analyzed, not only do they show the accuracy and performance of the proposed approach, but also the ways how SER can be employed in the Metaverse platforms.

Please use this identifier to cite or link to this item:

http://hdl.handle.net/10453/179007