One-Shot Learning-Based Animal Video Segmentation

Xue, T; Qiao, Y; Kong, H; Su, D; Pan, S; Rafique, K; Sukkarieh, S

One-Shot Learning-Based Animal Video Segmentation

Xue, T Qiao, Y Kong, H Su, D Pan, S

Rafique, K Sukkarieh, S

Permalink

Publisher:: IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
Publication Type:: Journal Article
Citation:: IEEE Transactions on Industrial Informatics, 2022, 18, (6), pp. 3799-3807
Issue Date:: 2022-06-01

Closed Access

	Filename	Description	Size
	One-Shot_Learning-Based_Animal_Video_Segmentation.pdf	Published version	4.27 MB	Adobe PDF	View/Open

Copyright Clearance Process

Recently Added
In Progress
Closed Access

This item is closed access and not available.

Full metadata record

Field	Value	Language
dc.contributor.author	Xue, T
dc.contributor.author	Qiao, Y
dc.contributor.author	Kong, H
dc.contributor.author	Su, D
dc.contributor.author	Pan, S https://orcid.org/0000-0003-0794-527X
dc.contributor.author	Rafique, K
dc.contributor.author	Sukkarieh, S
dc.date.accessioned	2022-04-26T06:16:44Z
dc.date.available	2022-04-26T06:16:44Z
dc.date.issued	2022-06-01
dc.identifier.citation	IEEE Transactions on Industrial Informatics, 2022, 18, (6), pp. 3799-3807
dc.identifier.issn	1551-3203
dc.identifier.issn	1941-0050
dc.identifier.uri	http://hdl.handle.net/10453/156614
dc.description.abstract	Deep learning-based video segmentation methods can offer a good performance after being trained on the large-scale pixel labeled datasets. However, a pixel-wise manual labeling of animal images is challenging and time consuming due to irregular contours and motion blur. To achieve desirable tradeoffs between the accuracy and speed, a novel one-shot learning-based approach is proposed in this article to segment animal video with only one labeled frame. The proposed approach consists of the following three main modules: guidance frame selection utilizes 'BubbleNet' to choose one frame for manual labeling, which can leverage the fine-tuning effects of the only labeled frame; Xception-based fully convolutional network localizes dense prediction using depthwise separable convolutions based on one single labeled frame; and postprocessing is used to remove outliers and sharpen object contours, which consists of two submodules-test time augmentation and conditional random field. Extensive experiments have been conducted on the DAVIS 2016 animal dataset. Our proposed video segmentation approach achieved mean intersection-over-union score of 89.5% on the DAVIS 2016 animal dataset with less run time, and outperformed the state-of-art methods (OSVOS and OSMN). The proposed one-shot learning-based approach achieves real-time and automatic segmentation of animals with only one labeled video frame. This can be potentially used further as a baseline for intelligent perception-based monitoring of animals and other domain-specific applications1.
dc.language	English
dc.publisher	IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
dc.relation.ispartof	IEEE Transactions on Industrial Informatics
dc.relation.isbasedon	10.1109/TII.2021.3117020
dc.rights	info:eu-repo/semantics/closedAccess
dc.subject	08 Information and Computing Sciences, 09 Engineering, 10 Technology
dc.subject.classification	Electrical & Electronic Engineering
dc.title	One-Shot Learning-Based Animal Video Segmentation
dc.type	Journal Article
utslib.citation.volume	18
utslib.for	08 Information and Computing Sciences
utslib.for	09 Engineering
utslib.for	10 Technology
pubs.organisational-group	/University of Technology Sydney
pubs.organisational-group	/University of Technology Sydney/Faculty of Engineering and Information Technology
utslib.copyright.status	closed_access	*
dc.date.updated	2022-04-26T06:16:34Z
pubs.issue	6
pubs.publication-status	Accepted
pubs.volume	18
utslib.citation.issue	6

Abstract:

Deep learning-based video segmentation methods can offer a good performance after being trained on the large-scale pixel labeled datasets. However, a pixel-wise manual labeling of animal images is challenging and time consuming due to irregular contours and motion blur. To achieve desirable tradeoffs between the accuracy and speed, a novel one-shot learning-based approach is proposed in this article to segment animal video with only one labeled frame. The proposed approach consists of the following three main modules: guidance frame selection utilizes 'BubbleNet' to choose one frame for manual labeling, which can leverage the fine-tuning effects of the only labeled frame; Xception-based fully convolutional network localizes dense prediction using depthwise separable convolutions based on one single labeled frame; and postprocessing is used to remove outliers and sharpen object contours, which consists of two submodules-test time augmentation and conditional random field. Extensive experiments have been conducted on the DAVIS 2016 animal dataset. Our proposed video segmentation approach achieved mean intersection-over-union score of 89.5% on the DAVIS 2016 animal dataset with less run time, and outperformed the state-of-art methods (OSVOS and OSMN). The proposed one-shot learning-based approach achieves real-time and automatic segmentation of animals with only one labeled video frame. This can be potentially used further as a baseline for intelligent perception-based monitoring of animals and other domain-specific applications1.

Please use this identifier to cite or link to this item:

http://hdl.handle.net/10453/156614