An Effective Ensemble Learning Framework for Affective Behaviour Analysis

Zhang, W; Qiu, F; Liu, C; Li, L; Du, H; Guo, T; Yu, X

An Effective Ensemble Learning Framework for Affective Behaviour Analysis

Zhang, W Qiu, F Liu, C Li, L Du, H Guo, T Yu, X

Permalink

Publisher:: IEEE
Publication Type:: Conference Proceeding
Citation:: 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2024, 00, pp. 4761-4772
Issue Date:: 2024-09-27

Closed Access

	Filename	Description	Size
	1756529.pdf	Published version	1.16 MB		View/Open

Copyright Clearance Process

Recently Added
In Progress
Closed Access

This item is closed access and not available.

Full metadata record

Field	Value	Language
dc.contributor.author	Zhang, W
dc.contributor.author	Qiu, F
dc.contributor.author	Liu, C
dc.contributor.author	Li, L
dc.contributor.author	Du, H
dc.contributor.author	Guo, T
dc.contributor.author	Yu, X https://orcid.org/0000-0002-0269-5649
dc.date	2024-06-17
dc.date.accessioned	2024-12-02T01:34:42Z
dc.date.available	2024-12-02T01:34:42Z
dc.date.issued	2024-09-27
dc.identifier.citation	2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2024, 00, pp. 4761-4772
dc.identifier.isbn	979-8-3503-6548-1
dc.identifier.issn	2160-7508
dc.identifier.issn	2160-7516
dc.identifier.uri	http://hdl.handle.net/10453/182256
dc.description.abstract	Affective Behavior Analysis aims to facilitate technology emotionally smart creating a world where devices can understand and react to our emotions as humans do To comprehensively evaluate the authenticity and applicability of emotional behavior analysis techniques in natural environments the 6th competition on Affective Behavior Analysis in the wild ABAW utilizes the Aff Wild2 Hume Vidmimic2 and C EXPR DB datasets to set up five competitive tracks i e Valence Arousal VA Estimation Expression EXPR Recognition Action Unit AU Detection Compound Expression CE Recognition and Emotional Mimicry Intensity EMI Estimation In this paper we present our method designs for VA estimation expression recognition and AU detection tracks Specifically our framework mainly includes three aspects 1 To achieve high quality facial feature representations we employ Masked Auto Encoder as the visual features extraction model and fine tune it with our facial dataset 2 Utilizing a transformer based feature fusion module to fully integrate emotional information provided by audio signals visual images and transcripts offering high quality expression features for the downstream tasks 3 Considering the complexity of the video collection scenes we conduct a more detailed dataset division based on scene characteristics and train the classifier for each scene Extensive experiments demonstrate the superiority of our designs Our work won the championship in the AU EXPR and VA tracks at the ABAW6 competition
dc.language	en
dc.publisher	IEEE
dc.relation.ispartof	2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)
dc.relation.ispartof	2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops
dc.relation.isbasedon	10.1109/cvprw63382.2024.00479
dc.rights	info:eu-repo/semantics/closedAccess
dc.title	An Effective Ensemble Learning Framework for Affective Behaviour Analysis
dc.type	Conference Proceeding
utslib.citation.volume	00
utslib.location.activity	Seattle, WA, USA
pubs.organisational-group	University of Technology Sydney
pubs.organisational-group	University of Technology Sydney/Faculty of Engineering and Information Technology
utslib.copyright.status	closed_access	*
dc.date.updated	2024-12-02T01:34:40Z
pubs.finish-date	2024-06-18
pubs.place-of-publication	Piscataway, USA
pubs.publication-status	Published
pubs.start-date	2024-06-17
pubs.volume	00
dc.location	Piscataway, USA

Abstract:

Affective Behavior Analysis aims to facilitate technology emotionally smart creating a world where devices can understand and react to our emotions as humans do To comprehensively evaluate the authenticity and applicability of emotional behavior analysis techniques in natural environments the 6th competition on Affective Behavior Analysis in the wild ABAW utilizes the Aff Wild2 Hume Vidmimic2 and C EXPR DB datasets to set up five competitive tracks i e Valence Arousal VA Estimation Expression EXPR Recognition Action Unit AU Detection Compound Expression CE Recognition and Emotional Mimicry Intensity EMI Estimation In this paper we present our method designs for VA estimation expression recognition and AU detection tracks Specifically our framework mainly includes three aspects 1 To achieve high quality facial feature representations we employ Masked Auto Encoder as the visual features extraction model and fine tune it with our facial dataset 2 Utilizing a transformer based feature fusion module to fully integrate emotional information provided by audio signals visual images and transcripts offering high quality expression features for the downstream tasks 3 Considering the complexity of the video collection scenes we conduct a more detailed dataset division based on scene characteristics and train the classifier for each scene Extensive experiments demonstrate the superiority of our designs Our work won the championship in the AU EXPR and VA tracks at the ABAW6 competition

Please use this identifier to cite or link to this item:

http://hdl.handle.net/10453/182256