A Generative Deep Learning Approach for Forensic Facial Reconstruction

Hargreaves, M; Ting, D; Bajan, S; Bhavnagri, K; Bassed, R; Chang, X

A Generative Deep Learning Approach for Forensic Facial Reconstruction

Hargreaves, M Ting, D Bajan, S Bhavnagri, K Bassed, R Chang, X

Permalink

Publisher:: IEEE
Publication Type:: Conference Proceeding
Citation:: DICTA 2021 - 2021 International Conference on Digital Image Computing: Techniques and Applications, 2021, 00, pp. 1-7
Issue Date:: 2021-01-01

Closed Access

	Filename	Description	Size
	A_Generative_Deep_Learning_Approach_for_Forensic_Facial_Reconstruction.pdf	Published version	8.93 MB	Adobe PDF	View/Open

Copyright Clearance Process

Recently Added
In Progress
Closed Access

This item is closed access and not available.

Full metadata record

Field	Value	Language
dc.contributor.author	Hargreaves, M
dc.contributor.author	Ting, D
dc.contributor.author	Bajan, S
dc.contributor.author	Bhavnagri, K
dc.contributor.author	Bassed, R
dc.contributor.author	Chang, X https://orcid.org/0000-0002-7778-8807
dc.date	2021-11-29
dc.date.accessioned	2022-06-10T19:30:59Z
dc.date.available	2022-06-10T19:30:59Z
dc.date.issued	2021-01-01
dc.identifier.citation	DICTA 2021 - 2021 International Conference on Digital Image Computing: Techniques and Applications, 2021, 00, pp. 1-7
dc.identifier.isbn	9781665417099
dc.identifier.uri	http://hdl.handle.net/10453/158075
dc.description.abstract	Forensic facial reconstruction currently relies on subjective manual methods to reconstruct a recognizable face from a skull. Automated approaches using algorithms and statistical norms have been able to reliably construct faces in real-time, but are unable to generate areas of the face that do not correlate strongly to the bone beneath, such as the eyes, lips and ears. Recent developments in deep learning have shown that generative models can produce realistic images indistinguishable from genuine human faces. Applying these techniques, we propose a generative deep learning solution to perform facial reconstruction directly from the bone with limited data and no background expertise. The model is trained on 665 3D Computed Tomography (CT) head scans that have been cleaned of noise, rotated to correct orientation and then filtered by density to find bone and soft tissue to be used as input and label respectively. It is trained with a combination of adversarial and VGGFace2 perceptual loss. The model is then compared to two baseline deep generative models and achieves a mIoU of 0.9410 and facial detection score of 88.32%. Results show the model is able to consistently generate accurate jaw and muscle structures. Additionally, it generates realistic ears, eyes and noses - features that are traditionally difficult to generate automatically with traditional techniques. Our model provides the basis for a complete, end-to-end solution for forensic facial reconstruction, with no prior knowledge or training on reconstruction techniques.
dc.language	en
dc.publisher	IEEE
dc.relation.ispartof	DICTA 2021 - 2021 International Conference on Digital Image Computing: Techniques and Applications
dc.relation.ispartof	Digital Image Computing: Techniques and Applications
dc.relation.isbasedon	10.1109/DICTA52665.2021.9647290
dc.rights	info:eu-repo/semantics/closedAccess
dc.title	A Generative Deep Learning Approach for Forensic Facial Reconstruction
dc.type	Conference Proceeding
utslib.citation.volume	00
utslib.location.activity	Gold Coast, Australia
pubs.organisational-group	/University of Technology Sydney
pubs.organisational-group	/University of Technology Sydney/Faculty of Engineering and Information Technology
pubs.organisational-group	/University of Technology Sydney/Faculty of Engineering and Information Technology/School of Computer Science
utslib.copyright.status	closed_access	*
pubs.consider-herdc	false
dc.date.updated	2022-06-10T19:30:52Z
pubs.finish-date	2021-12-01
pubs.place-of-publication	Piscataway, USA
pubs.publication-status	Published
pubs.start-date	2021-11-29
pubs.volume	00
dc.location	Piscataway, USA

Abstract:

Forensic facial reconstruction currently relies on subjective manual methods to reconstruct a recognizable face from a skull. Automated approaches using algorithms and statistical norms have been able to reliably construct faces in real-time, but are unable to generate areas of the face that do not correlate strongly to the bone beneath, such as the eyes, lips and ears. Recent developments in deep learning have shown that generative models can produce realistic images indistinguishable from genuine human faces. Applying these techniques, we propose a generative deep learning solution to perform facial reconstruction directly from the bone with limited data and no background expertise. The model is trained on 665 3D Computed Tomography (CT) head scans that have been cleaned of noise, rotated to correct orientation and then filtered by density to find bone and soft tissue to be used as input and label respectively. It is trained with a combination of adversarial and VGGFace2 perceptual loss. The model is then compared to two baseline deep generative models and achieves a mIoU of 0.9410 and facial detection score of 88.32%. Results show the model is able to consistently generate accurate jaw and muscle structures. Additionally, it generates realistic ears, eyes and noses - features that are traditionally difficult to generate automatically with traditional techniques. Our model provides the basis for a complete, end-to-end solution for forensic facial reconstruction, with no prior knowledge or training on reconstruction techniques.

Please use this identifier to cite or link to this item:

http://hdl.handle.net/10453/158075