GEME: Dual-stream multi-task GEnder-based micro-expression recognition

Nie, X; Takalkar, MA; Duan, M; Zhang, H; Xu, M

GEME: Dual-stream multi-task GEnder-based micro-expression recognition

Nie, X Takalkar, MA Duan, M Zhang, H

Xu, M

Permalink

Publisher:: ELSEVIER
Publication Type:: Journal Article
Citation:: Neurocomputing, 2021, 427, pp. 13-28
Issue Date:: 2021-02-28

Closed Access

	Filename	Description	Size
	1-s2.0-S0925231220316957-main.pdf	Published version	2.66 MB	Adobe PDF	View/Open

Copyright Clearance Process

Recently Added
In Progress
Closed Access

This item is closed access and not available.

Full metadata record

Field	Value	Language
dc.contributor.author	Nie, X
dc.contributor.author	Takalkar, MA
dc.contributor.author	Duan, M
dc.contributor.author	Zhang, H https://orcid.org/0000-0002-0021-3634
dc.contributor.author	Xu, M https://orcid.org/0000-0001-9581-8849
dc.date.accessioned	2022-04-30T03:47:11Z
dc.date.available	2022-04-30T03:47:11Z
dc.date.issued	2021-02-28
dc.identifier.citation	Neurocomputing, 2021, 427, pp. 13-28
dc.identifier.issn	0925-2312
dc.identifier.issn	1872-8286
dc.identifier.uri	http://hdl.handle.net/10453/156852
dc.description.abstract	Recognition of micro-expressions remains a topic of concern considering its brief span and low intensity. This issue is addressed through convolutional neural networks (CNNs) by developing multi-task learning (MTL) method to effectively leverage a side task: gender detection. A dual-stream multi-task framework called GEME is introduced that recognises micro-expressions by incorporating unique gender characteristics and subsequently improves the micro-expression recognition accuracy. This research aims to examine how gender differences influence the way micro-expressions are displayed. The current study proves that selecting relevant features of micro-expressions distinctive to the gender and added to the micro-expression features improves the micro-expression recognition accuracy. This network learns gender-specific features and micro-expression features and adds them together to learn the combination of shared and task-specific representations. A multi-class focal loss is used to mitigate the class imbalance issue by down-weighing the easy samples and concentrate more on misclassified samples. The Class-Balanced (CB) focal loss is also implemented for a better class balancing during Leave-One-Subject-Out (LOSO) validations where CB loss re-balances and re-weights the loss. The experimental results on three widely used databases demonstrate the improved performance of the proposed network and achieve comparable results with the state-of-the-art methods.
dc.language	English
dc.publisher	ELSEVIER
dc.relation.ispartof	Neurocomputing
dc.relation.isbasedon	10.1016/j.neucom.2020.10.082
dc.rights	info:eu-repo/semantics/closedAccess
dc.subject	08 Information and Computing Sciences, 09 Engineering, 17 Psychology and Cognitive Sciences
dc.subject.classification	Artificial Intelligence & Image Processing
dc.title	GEME: Dual-stream multi-task GEnder-based micro-expression recognition
dc.type	Journal Article
utslib.citation.volume	427
utslib.for	08 Information and Computing Sciences
utslib.for	09 Engineering
utslib.for	17 Psychology and Cognitive Sciences
pubs.organisational-group	/University of Technology Sydney
pubs.organisational-group	/University of Technology Sydney/Faculty of Engineering and Information Technology
pubs.organisational-group	/University of Technology Sydney/Strength - INEXT - Innovation in IT Services and Applications
pubs.organisational-group	/University of Technology Sydney/Strength - GBDTC - Global Big Data Technologies
pubs.organisational-group	/University of Technology Sydney/Faculty of Engineering and Information Technology/School of Electrical and Data Engineering
pubs.organisational-group	/University of Technology Sydney/Faculty of Engineering and Information Technology/School of Computer Science
utslib.copyright.status	closed_access	*
dc.date.updated	2022-04-30T03:47:08Z
pubs.publication-status	Published
pubs.volume	427

Abstract:

Recognition of micro-expressions remains a topic of concern considering its brief span and low intensity. This issue is addressed through convolutional neural networks (CNNs) by developing multi-task learning (MTL) method to effectively leverage a side task: gender detection. A dual-stream multi-task framework called GEME is introduced that recognises micro-expressions by incorporating unique gender characteristics and subsequently improves the micro-expression recognition accuracy. This research aims to examine how gender differences influence the way micro-expressions are displayed. The current study proves that selecting relevant features of micro-expressions distinctive to the gender and added to the micro-expression features improves the micro-expression recognition accuracy. This network learns gender-specific features and micro-expression features and adds them together to learn the combination of shared and task-specific representations. A multi-class focal loss is used to mitigate the class imbalance issue by down-weighing the easy samples and concentrate more on misclassified samples. The Class-Balanced (CB) focal loss is also implemented for a better class balancing during Leave-One-Subject-Out (LOSO) validations where CB loss re-balances and re-weights the loss. The experimental results on three widely used databases demonstrate the improved performance of the proposed network and achieve comparable results with the state-of-the-art methods.

Please use this identifier to cite or link to this item:

http://hdl.handle.net/10453/156852