Cluster-Phys: Facial Clues Clustering Towards Efficient Remote Physiological Measurement

Qia, N; Li, K; Guo, D; Hu, B; Wang, M

Cluster-Phys: Facial Clues Clustering Towards Efficient Remote Physiological Measurement

Qia, N Li, K

Guo, D Hu, B Wang, M

Permalink

Publisher:: ACM
Publication Type:: Conference Proceeding
Citation:: MM 2024 - Proceedings of the 32nd ACM International Conference on Multimedia, 2024, pp. 330-339
Issue Date:: 2024-10-28

Closed Access

	Filename	Description	Size
	3664647.3680670.pdf	Published version	2.99 MB	Adobe PDF	View/Open

Copyright Clearance Process

Recently Added
In Progress
Closed Access

This item is closed access and not available.

Full metadata record

Field	Value	Language
dc.contributor.author	Qia, N
dc.contributor.author	Li, K https://orcid.org/0000-0001-5083-2145
dc.contributor.author	Guo, D
dc.contributor.author	Hu, B
dc.contributor.author	Wang, M
dc.date	2024-10-28
dc.date.accessioned	2025-03-19T00:26:36Z
dc.date.available	2025-03-19T00:26:36Z
dc.date.issued	2024-10-28
dc.identifier.citation	MM 2024 - Proceedings of the 32nd ACM International Conference on Multimedia, 2024, pp. 330-339
dc.identifier.isbn	979-8-4007-0686-8
dc.identifier.uri	http://hdl.handle.net/10453/185981
dc.description.abstract	Remote photoplethysmography (rPPG) measurement aims to estimate physiological signals by analyzing subtle skin color changes induced by heartbeats in facial videos. Existing methods primarily rely on the fundamental video frame features or vanilla facial ROI (region of interest) features. Recognizing the varying light absorption and reactions of different facial regions over time, we adopt a new perspective to conduct a more fine-grained exploration of the key clues present in different facial regions within each frame and across temporal frames. Concretely, we propose a novel clustering-driven remote physiological measurement framework called Cluster-Phys, which employs a facial ROI prototypical clustering module to adaptively cluster the representative facial ROI features as facial prototypes and then update facial prototypes with highly semantic correlated base ROI features. In this way, our approach can mine facial clues from a more compact and informative prototype level rather than the conventional video/ROI level. Furthermore, we also propose a spatial-temporal prototype interaction module to learn facial prototype correlation from both spatial (across prototypes) and temporal (within prototype) perspectives. Extensive experiments are conducted on both intra-dataset and cross-dataset tests. The results show that our Cluster-Phys achieves significant performance improvement with less computation consumption. The source code will be available at https://github.com/VUT-HFUT/ClusterPhys.
dc.language	en
dc.publisher	ACM
dc.relation.ispartof	MM 2024 - Proceedings of the 32nd ACM International Conference on Multimedia
dc.relation.ispartof	ACM International Conference on Multimedia
dc.relation.isbasedon	10.1145/3664647.3680670
dc.rights	info:eu-repo/semantics/closedAccess
dc.title	Cluster-Phys: Facial Clues Clustering Towards Efficient Remote Physiological Measurement
dc.type	Conference Proceeding
utslib.location.activity	Melbourne VIC Australia
pubs.organisational-group	University of Technology Sydney
pubs.organisational-group	University of Technology Sydney/Faculty of Engineering and Information Technology
utslib.copyright.status	closed_access	*
pubs.consider-herdc	false
dc.date.updated	2025-03-19T00:26:33Z
pubs.finish-date	2024-11-01
pubs.place-of-publication	USA
pubs.publication-status	Published
pubs.start-date	2024-10-28
dc.location	USA

Abstract:

Remote photoplethysmography (rPPG) measurement aims to estimate physiological signals by analyzing subtle skin color changes induced by heartbeats in facial videos. Existing methods primarily rely on the fundamental video frame features or vanilla facial ROI (region of interest) features. Recognizing the varying light absorption and reactions of different facial regions over time, we adopt a new perspective to conduct a more fine-grained exploration of the key clues present in different facial regions within each frame and across temporal frames. Concretely, we propose a novel clustering-driven remote physiological measurement framework called Cluster-Phys, which employs a facial ROI prototypical clustering module to adaptively cluster the representative facial ROI features as facial prototypes and then update facial prototypes with highly semantic correlated base ROI features. In this way, our approach can mine facial clues from a more compact and informative prototype level rather than the conventional video/ROI level. Furthermore, we also propose a spatial-temporal prototype interaction module to learn facial prototype correlation from both spatial (across prototypes) and temporal (within prototype) perspectives. Extensive experiments are conducted on both intra-dataset and cross-dataset tests. The results show that our Cluster-Phys achieves significant performance improvement with less computation consumption. The source code will be available at https://github.com/VUT-HFUT/ClusterPhys.

Please use this identifier to cite or link to this item:

http://hdl.handle.net/10453/185981