Selective combination of visual and thermal imaging for resilient localization in adverse conditions: Day and night, smoke and fire

Brunner, C; Peynot, T; Vidal-Calleja, T; Underwood, J

Selective combination of visual and thermal imaging for resilient localization in adverse conditions: Day and night, smoke and fire

Brunner, C Peynot, T Vidal-Calleja, T

Underwood, J

Permalink

Publication Type:: Journal Article
Citation:: Journal of Field Robotics, 2013, 30 (4), pp. 641 - 666
Issue Date:: 2013-07-01

Closed Access

	Filename	Description	Size
	2012004473OK.pdf		1.4 MB	Adobe PDF	View/Open

Copyright Clearance Process

Recently Added
In Progress
Closed Access

This item is closed access and not available.

Full metadata record

Field	Value	Language
dc.contributor.author	Brunner, C	en_US
dc.contributor.author	Peynot, T	en_US
dc.contributor.author	Vidal-Calleja, T https://orcid.org/0000-0002-5763-9644	en_US
dc.contributor.author	Underwood, J	en_US
dc.date.issued	2013-07-01	en_US
dc.identifier.citation	Journal of Field Robotics, 2013, 30 (4), pp. 641 - 666	en_US
dc.identifier.issn	1556-4959	en_US
dc.identifier.uri	http://hdl.handle.net/10453/23591
dc.description.abstract	Long-term autonomy in robotics requires perception systems that are resilient to unusual but realistic conditions that will eventually occur during extended missions. For example, unmanned ground vehicles (UGVs) need to be capable of operating safely in adverse and low-visibility conditions, such as at night or in the presence of smoke. The key to a resilient UGV perception system lies in the use of multiple sensor modalities, e.g., operating at different frequencies of the electromagnetic spectrum, to compensate for the limitations of a single sensor type. In this paper, visual and infrared imaging are combined in a Visual-SLAM algorithm to achieve localization. We propose to evaluate the quality of data provided by each sensor modality prior to data combination. This evaluation is used to discard low-quality data, i.e., data most likely to induce large localization errors. In this way, perceptual failures are anticipated and mitigated. An extensive experimental evaluation is conducted on data sets collected with a UGV in a range of environments and adverse conditions, including the presence of smoke (obstructing the visual camera), fire, extreme heat (saturating the infrared camera), low-light conditions (dusk), and at night with sudden variations of artificial light. A total of 240 trajectory estimates are obtained using five different variations of data sources and data combination strategies in the localization method. In particular, the proposed approach for selective data combination is compared to methods using a single sensor type or combining both modalities without preselection. We show that the proposed framework allows for camera-based localization resilient to a large range of low-visibility conditions. © 2013 Wiley Periodicals, Inc.	en_US
dc.relation.ispartof	Journal of Field Robotics	en_US
dc.relation.isbasedon	10.1002/rob.21464	en_US
dc.subject.classification	Industrial Engineering & Automation	en_US
dc.title	Selective combination of visual and thermal imaging for resilient localization in adverse conditions: Day and night, smoke and fire	en_US
dc.type	Journal Article
utslib.citation.volume	4	en_US
utslib.citation.volume	30	en_US
utslib.for	0801 Artificial Intelligence and Image Processing	en_US
utslib.for	0906 Electrical and Electronic Engineering	en_US
utslib.for	0913 Mechanical Engineering	en_US
pubs.embargo.period	Not known	en_US
pubs.organisational-group	/University of Technology Sydney
pubs.organisational-group	/University of Technology Sydney/Faculty of Engineering and Information Technology
pubs.organisational-group	/University of Technology Sydney/Faculty of Engineering and Information Technology/School of Mechanical and Mechatronic Engineering
pubs.organisational-group	/University of Technology Sydney/Strength - CAS - Centre for Autonomous Systems
utslib.copyright.status	closed_access
pubs.issue	4	en_US
pubs.publication-status	Published	en_US
pubs.volume	30	en_US

Abstract:

Long-term autonomy in robotics requires perception systems that are resilient to unusual but realistic conditions that will eventually occur during extended missions. For example, unmanned ground vehicles (UGVs) need to be capable of operating safely in adverse and low-visibility conditions, such as at night or in the presence of smoke. The key to a resilient UGV perception system lies in the use of multiple sensor modalities, e.g., operating at different frequencies of the electromagnetic spectrum, to compensate for the limitations of a single sensor type. In this paper, visual and infrared imaging are combined in a Visual-SLAM algorithm to achieve localization. We propose to evaluate the quality of data provided by each sensor modality prior to data combination. This evaluation is used to discard low-quality data, i.e., data most likely to induce large localization errors. In this way, perceptual failures are anticipated and mitigated. An extensive experimental evaluation is conducted on data sets collected with a UGV in a range of environments and adverse conditions, including the presence of smoke (obstructing the visual camera), fire, extreme heat (saturating the infrared camera), low-light conditions (dusk), and at night with sudden variations of artificial light. A total of 240 trajectory estimates are obtained using five different variations of data sources and data combination strategies in the localization method. In particular, the proposed approach for selective data combination is compared to methods using a single sensor type or combining both modalities without preselection. We show that the proposed framework allows for camera-based localization resilient to a large range of low-visibility conditions. © 2013 Wiley Periodicals, Inc.

Please use this identifier to cite or link to this item:

http://hdl.handle.net/10453/23591