Do We Really Need Frame-by-Frame Annotation Datasets for Object Tracking?

Hu, L; Huang, S; Wang, S; Liu, W; Ning, J

Do We Really Need Frame-by-Frame Annotation Datasets for Object Tracking?

Hu, L Huang, S Wang, S Liu, W

Ning, J

Permalink

Publisher:: ACM
Publication Type:: Conference Proceeding
Citation:: MM 2021 - Proceedings of the 29th ACM International Conference on Multimedia, 2021, pp. 4949-4957
Issue Date:: 2021-10-17

Closed Access

	Filename	Description	Size
	3474085.3475365.pdf	Published version	8.32 MB	Adobe PDF	View/Open

Copyright Clearance Process

Recently Added
In Progress
Closed Access

This item is closed access and not available.

Full metadata record

Field	Value	Language
dc.contributor.author	Hu, L
dc.contributor.author	Huang, S
dc.contributor.author	Wang, S
dc.contributor.author	Liu, W https://orcid.org/0000-0002-3003-1313
dc.contributor.author	Ning, J
dc.date.accessioned	2022-06-10T20:45:09Z
dc.date.available	2022-06-10T20:45:09Z
dc.date.issued	2021-10-17
dc.identifier.citation	MM 2021 - Proceedings of the 29th ACM International Conference on Multimedia, 2021, pp. 4949-4957
dc.identifier.isbn	9781450386517
dc.identifier.uri	http://hdl.handle.net/10453/158079
dc.description.abstract	There has been an increasing emphasis on building large-scale datasets as the driver of deep learning-based trackers' success. However, accurately annotating tracking data is highly labor-intensive and expensive, making it infeasible in real-world applications. In this study, we investigate the necessity of large-scale training data to ensure tracking algorithms' performance. To this end, we introduce a FAT (Few-Annotation Tracking) benchmark constructed by sampling one or a few frames per video from some existing tracking datasets. The proposed dataset can be used to evaluate the effectiveness of tracking algorithms considering data efficiency and new data augmentation approaches for object tracking. We further present AMMC (Augmentation by Mimicking Motion Change), a data augmentation strategy that enables learning high-performing trackers using small-scale datasets. AMMC first cuts out the tracked targets and performs a sequence of transformations to simulate the possible change by object motion. Then the transformed targets are pasted on the inpainted background images and further conjointly augmented to mimic variability caused by camera motion. Compared with standard augmentation methods, AMMC explicitly considers tracking data characteristics, which synthesizes more valid data for object tracking. We extensively evaluate our approach with two popular trackers on the FAT datasets. Experiments show that our method allows these trackers to even trained on a dataset requiring much less annotation to achieve comparable or even better performance to those on the full-annotation dataset. The results imply complete video annotation might not be necessary for object tracking if leveraging motion-driven data augmentations during training.
dc.language	en
dc.publisher	ACM
dc.relation.ispartof	MM 2021 - Proceedings of the 29th ACM International Conference on Multimedia
dc.relation.ispartof	Proceedings of the 29th ACM International Conference on Multimedia
dc.relation.isbasedon	10.1145/3474085.3475365
dc.rights	info:eu-repo/semantics/closedAccess
dc.title	Do We Really Need Frame-by-Frame Annotation Datasets for Object Tracking?
dc.type	Conference Proceeding
pubs.organisational-group	/University of Technology Sydney
pubs.organisational-group	/University of Technology Sydney/Faculty of Engineering and Information Technology
pubs.organisational-group	/University of Technology Sydney/Strength - AAI - Advanced Analytics Institute Research Centre
pubs.organisational-group	/University of Technology Sydney/Faculty of Engineering and Information Technology/School of Computer Science
utslib.copyright.status	closed_access	*
dc.date.updated	2022-06-10T20:45:04Z
pubs.publication-status	Published

Abstract:

There has been an increasing emphasis on building large-scale datasets as the driver of deep learning-based trackers' success. However, accurately annotating tracking data is highly labor-intensive and expensive, making it infeasible in real-world applications. In this study, we investigate the necessity of large-scale training data to ensure tracking algorithms' performance. To this end, we introduce a FAT (Few-Annotation Tracking) benchmark constructed by sampling one or a few frames per video from some existing tracking datasets. The proposed dataset can be used to evaluate the effectiveness of tracking algorithms considering data efficiency and new data augmentation approaches for object tracking. We further present AMMC (Augmentation by Mimicking Motion Change), a data augmentation strategy that enables learning high-performing trackers using small-scale datasets. AMMC first cuts out the tracked targets and performs a sequence of transformations to simulate the possible change by object motion. Then the transformed targets are pasted on the inpainted background images and further conjointly augmented to mimic variability caused by camera motion. Compared with standard augmentation methods, AMMC explicitly considers tracking data characteristics, which synthesizes more valid data for object tracking. We extensively evaluate our approach with two popular trackers on the FAT datasets. Experiments show that our method allows these trackers to even trained on a dataset requiring much less annotation to achieve comparable or even better performance to those on the full-annotation dataset. The results imply complete video annotation might not be necessary for object tracking if leveraging motion-driven data augmentations during training.

Please use this identifier to cite or link to this item:

http://hdl.handle.net/10453/158079