Graph-Based Facial Affect Analysis: A Review

Liu, Y; Zhang, X; Li, Y; Zhou, J; Li, X; Zhao, G

Graph-Based Facial Affect Analysis: A Review

Liu, Y Zhang, X Li, Y Zhou, J

Li, X Zhao, G

Permalink

Publisher:: IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
Publication Type:: Journal Article
Citation:: IEEE Transactions on Affective Computing, 2023, 14, (4), pp. 2657-2677
Issue Date:: 2023-10-01

Open Access

Copyright Clearance Process

Recently Added
In Progress
Open Access

This item is open access.

Adobe PDF

Download Accepted versionAdobe PDF (3.36 MB)

View on publisher's site

View statistics

Full metadata record

Field	Value	Language
dc.contributor.author	Liu, Y
dc.contributor.author	Zhang, X
dc.contributor.author	Li, Y
dc.contributor.author	Zhou, J https://orcid.org/0000-0002-6620-604X
dc.contributor.author	Li, X
dc.contributor.author	Zhao, G
dc.date.accessioned	2024-03-27T21:57:09Z
dc.date.available	2024-03-27T21:57:09Z
dc.date.issued	2023-10-01
dc.identifier.citation	IEEE Transactions on Affective Computing, 2023, 14, (4), pp. 2657-2677
dc.identifier.issn	1949-3045
dc.identifier.issn	1949-3045
dc.identifier.uri	http://hdl.handle.net/10453/177284
dc.description.abstract	As one of the most important affective signals, facial affect analysis (FAA) is essential for developing human-computer interaction systems. Early methods focus on extracting appearance and geometry features associated with human affects while ignoring the latent semantic information among individual facial changes, leading to limited performance and generalization. Recent work attempts to establish a graph-based representation to model these semantic relationships and develop frameworks to leverage them for various FAA tasks. This article provides a comprehensive review of graph-based FAA, including the evolution of algorithms and their applications. First, the FAA background knowledge is introduced, especially on the role of the graph. We then discuss approaches widely used for graph-based affective representation in literature and show a trend towards graph construction. For the relational reasoning in graph-based FAA, existing studies are categorized according to their non-deep or deep learning methods, emphasizing the latest graph neural networks. Performance comparisons of the state-of-the-art graph-based FAA methods are also summarized. Finally, we discuss the challenges and potential directions. As far as we know, this is the first survey of graph-based FAA methods. Our findings can serve as a reference for future research in this field.
dc.language	English
dc.publisher	IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
dc.relation.ispartof	IEEE Transactions on Affective Computing
dc.relation.isbasedon	10.1109/TAFFC.2022.3215918
dc.rights	info:eu-repo/semantics/openAccess
dc.subject	0801 Artificial Intelligence and Image Processing, 0806 Information Systems, 1702 Cognitive Sciences
dc.subject.classification	4602 Artificial intelligence
dc.subject.classification	4603 Computer vision and multimedia computation
dc.subject.classification	4608 Human-centred computing
dc.title	Graph-Based Facial Affect Analysis: A Review
dc.type	Journal Article
utslib.citation.volume	14
utslib.for	0801 Artificial Intelligence and Image Processing
utslib.for	0806 Information Systems
utslib.for	1702 Cognitive Sciences
pubs.organisational-group	University of Technology Sydney
pubs.organisational-group	University of Technology Sydney/Faculty of Engineering and Information Technology
pubs.organisational-group	University of Technology Sydney/Faculty of Engineering and Information Technology/School of Computer Science
utslib.copyright.status	open_access	*
dc.date.updated	2024-03-27T21:57:06Z
pubs.issue	4
pubs.publication-status	Published
pubs.volume	14
utslib.citation.issue	4

Abstract:

As one of the most important affective signals, facial affect analysis (FAA) is essential for developing human-computer interaction systems. Early methods focus on extracting appearance and geometry features associated with human affects while ignoring the latent semantic information among individual facial changes, leading to limited performance and generalization. Recent work attempts to establish a graph-based representation to model these semantic relationships and develop frameworks to leverage them for various FAA tasks. This article provides a comprehensive review of graph-based FAA, including the evolution of algorithms and their applications. First, the FAA background knowledge is introduced, especially on the role of the graph. We then discuss approaches widely used for graph-based affective representation in literature and show a trend towards graph construction. For the relational reasoning in graph-based FAA, existing studies are categorized according to their non-deep or deep learning methods, emphasizing the latest graph neural networks. Performance comparisons of the state-of-the-art graph-based FAA methods are also summarized. Finally, we discuss the challenges and potential directions. As far as we know, this is the first survey of graph-based FAA methods. Our findings can serve as a reference for future research in this field.

Please use this identifier to cite or link to this item:

http://hdl.handle.net/10453/177284