Supervision by Registration and Triangulation for Landmark Detection.

Dong, X; Yang, Y; Wei, S-E; Weng, X; Sheikh, Y; Yu, S-I

Supervision by Registration and Triangulation for Landmark Detection.

Dong, X

Yang, Y

Wei, S-E Weng, X Sheikh, Y Yu, S-I

Permalink

Publisher:: IEEE COMPUTER SOC
Publication Type:: Journal Article
Citation:: IEEE transactions on pattern analysis and machine intelligence, 2021, PP, (10), pp. 3681-3694
Issue Date:: 2021-01-01

Open Access

Copyright Clearance Process

Recently Added
In Progress
Open Access

This item is open access.

The embargo period expires on 31 Oct 2023

Adobe PDF

Download Accepted versionAdobe PDF (8.18 MB)

View on publisher's site

View statistics

Full metadata record

Field	Value	Language
dc.contributor.author	Dong, X https://orcid.org/0000-0001-9272-1590
dc.contributor.author	Yang, Y https://orcid.org/0000-0002-0512-880X
dc.contributor.author	Wei, S-E
dc.contributor.author	Weng, X
dc.contributor.author	Sheikh, Y
dc.contributor.author	Yu, S-I
dc.date.accessioned	2022-05-09T05:46:02Z
dc.date.available	2022-05-09T05:46:02Z
dc.date.issued	2021-01-01
dc.identifier.citation	IEEE transactions on pattern analysis and machine intelligence, 2021, PP, (10), pp. 3681-3694
dc.identifier.issn	0162-8828
dc.identifier.issn	1939-3539
dc.identifier.uri	http://hdl.handle.net/10453/157158
dc.description.abstract	We present Supervision by Registration and Triangulation (SRT), an unsupervised approach that utilizes unlabeled multi-view video to improve the accuracy and precision of landmark detectors. Being able to utilize unlabeled data enables our detectors to learn from massive amounts of unlabeled data freely available and not be limited by the quality and quantity of manual human annotations. To utilize unlabeled data, there are two key observations: (1) the detections of the same landmark in adjacent frames should be coherent with registration, i.e., optical flow. (2) the detections of the same landmark in multiple synchronized and calibrated views should correspond to a single 3D point, i.e., multi-view consistency. Registration and multi-view consistency are sources of supervision that do not require manual labeling, thus it can be leveraged to augment existing training data during detector training. End-to-end training is made possible by differentiable registration and 3D triangulation modules. Experiments with 11 datasets and a newly proposed metric to measure precision demonstrate accuracy and precision improvements in landmark detection on both images and video.
dc.format	Print-Electronic
dc.language	eng
dc.publisher	IEEE COMPUTER SOC
dc.relation.ispartof	IEEE transactions on pattern analysis and machine intelligence
dc.relation.isbasedon	10.1109/tpami.2020.2983935
dc.rights	info:eu-repo/semantics/embargoedAccess
dc.subject	0801 Artificial Intelligence and Image Processing, 0806 Information Systems, 0906 Electrical and Electronic Engineering
dc.subject.classification	Artificial Intelligence & Image Processing
dc.title	Supervision by Registration and Triangulation for Landmark Detection.
dc.type	Journal Article
utslib.citation.volume	PP
utslib.location.activity	United States
utslib.for	0801 Artificial Intelligence and Image Processing
utslib.for	0806 Information Systems
utslib.for	0906 Electrical and Electronic Engineering
pubs.organisational-group	/University of Technology Sydney
pubs.organisational-group	/University of Technology Sydney/Faculty of Engineering and Information Technology
pubs.organisational-group	/University of Technology Sydney/Strength - AAII - Australian Artificial Intelligence Institute
pubs.organisational-group	/University of Technology Sydney/Faculty of Engineering and Information Technology/School of Computer Science
utslib.copyright.status	open_access	*
pubs.consider-herdc	false
utslib.copyright.embargo	2023-10-31T00:00:00+1000Z
dc.date.updated	2022-05-09T05:45:35Z
pubs.issue	10
pubs.publication-status	Published
pubs.volume	PP
utslib.citation.issue	10

Abstract:

We present Supervision by Registration and Triangulation (SRT), an unsupervised approach that utilizes unlabeled multi-view video to improve the accuracy and precision of landmark detectors. Being able to utilize unlabeled data enables our detectors to learn from massive amounts of unlabeled data freely available and not be limited by the quality and quantity of manual human annotations. To utilize unlabeled data, there are two key observations: (1) the detections of the same landmark in adjacent frames should be coherent with registration, i.e., optical flow. (2) the detections of the same landmark in multiple synchronized and calibrated views should correspond to a single 3D point, i.e., multi-view consistency. Registration and multi-view consistency are sources of supervision that do not require manual labeling, thus it can be leveraged to augment existing training data during detector training. End-to-end training is made possible by differentiable registration and 3D triangulation modules. Experiments with 11 datasets and a newly proposed metric to measure precision demonstrate accuracy and precision improvements in landmark detection on both images and video.

Please use this identifier to cite or link to this item:

http://hdl.handle.net/10453/157158