Instance as Identity: A Generic Online Paradigm for Video Instance Segmentation

Zhu, F; Yang, Z; Yu, X; Yang, Y; Wei, Y

Instance as Identity: A Generic Online Paradigm for Video Instance Segmentation

Zhu, F Yang, Z Yu, X

Yang, Y

Wei, Y

Permalink

Publisher:: Springer Nature
Publication Type:: Chapter
Citation:: Computer Vision – ECCV 2022, 2022, 13689 LNCS, pp. 524-540
Issue Date:: 2022-01-01

Closed Access

	Filename	Description	Size
	978-3-031-19818-2_30.pdf	Published version	1.31 MB	Adobe PDF	View/Open

Copyright Clearance Process

Recently Added
In Progress
Closed Access

This item is closed access and not available.

Full metadata record

Field	Value	Language
dc.contributor.author	Zhu, F
dc.contributor.author	Yang, Z
dc.contributor.author	Yu, X https://orcid.org/0000-0002-0269-5649
dc.contributor.author	Yang, Y https://orcid.org/0000-0002-0512-880X
dc.contributor.author	Wei, Y
dc.date.accessioned	2023-01-27T04:09:47Z
dc.date.available	2023-01-27T04:09:47Z
dc.date.issued	2022-01-01
dc.identifier.citation	Computer Vision – ECCV 2022, 2022, 13689 LNCS, pp. 524-540
dc.identifier.isbn	9783031198175
dc.identifier.uri	http://hdl.handle.net/10453/165529
dc.description.abstract	Modeling temporal information for both detection and tracking in a unified framework has been proved a promising solution to video instance segmentation (VIS). However, how to effectively incorporate the temporal information into an online model remains an open problem. In this work, we propose a new online VIS paradigm named Instance As Identity (IAI), which models temporal information for both detection and tracking in an efficient way. In detail, IAI employs a novel identification module to predict identification number for tracking instances explicitly. For passing temporal information cross frame, IAI utilizes an association module which combines current features and past embeddings. Notably, IAI can be integrated with different image models. We conduct extensive experiments on three VIS benchmarks. IAI outperforms all the online competitors on YouTube-VIS-2019 (ResNet-101 41.9 mAP) and YouTube-VIS-2021 (ResNet-50 37.7 mAP). Surprisingly, on the more challenging OVIS, IAI achieves SOTA performance (20.3 mAP). Code is available at https://github.com/zfonemore/IAI.
dc.language	en
dc.publisher	Springer Nature
dc.relation.ispartof	Computer Vision – ECCV 2022
dc.relation.ispartofseries	Lecture Notes in Computer Science
dc.relation.isbasedon	10.1007/978-3-031-19818-2_30
dc.rights	info:eu-repo/semantics/closedAccess
dc.subject.classification	Artificial Intelligence & Image Processing
dc.title	Instance as Identity: A Generic Online Paradigm for Video Instance Segmentation
dc.type	Chapter
utslib.citation.volume	13689 LNCS
pubs.organisational-group	/University of Technology Sydney
pubs.organisational-group	/University of Technology Sydney/Faculty of Engineering and Information Technology
pubs.organisational-group	/University of Technology Sydney/Strength - AAII - Australian Artificial Intelligence Institute
utslib.copyright.status	closed_access	*
dc.date.updated	2023-01-27T04:09:44Z
pubs.publication-status	Published
pubs.volume	13689 LNCS

Abstract:

Modeling temporal information for both detection and tracking in a unified framework has been proved a promising solution to video instance segmentation (VIS). However, how to effectively incorporate the temporal information into an online model remains an open problem. In this work, we propose a new online VIS paradigm named Instance As Identity (IAI), which models temporal information for both detection and tracking in an efficient way. In detail, IAI employs a novel identification module to predict identification number for tracking instances explicitly. For passing temporal information cross frame, IAI utilizes an association module which combines current features and past embeddings. Notably, IAI can be integrated with different image models. We conduct extensive experiments on three VIS benchmarks. IAI outperforms all the online competitors on YouTube-VIS-2019 (ResNet-101 41.9 mAP) and YouTube-VIS-2021 (ResNet-50 37.7 mAP). Surprisingly, on the more challenging OVIS, IAI achieves SOTA performance (20.3 mAP). Code is available at https://github.com/zfonemore/IAI.

Please use this identifier to cite or link to this item:

http://hdl.handle.net/10453/165529