DifFormer: Multi-Resolutional Differencing Transformer With Dynamic Ranging for Time Series Analysis.

Li, B; Cui, W; Zhang, L; Zhu, C; Wang, W; Tsang, IW; Zhou, JT

DifFormer: Multi-Resolutional Differencing Transformer With Dynamic Ranging for Time Series Analysis.

Li, B Cui, W Zhang, L Zhu, C Wang, W Tsang, IW Zhou, JT

Permalink

Publisher:: IEEE COMPUTER SOC
Publication Type:: Journal Article
Citation:: IEEE Trans Pattern Anal Mach Intell, 2023, 45, (11), pp. 13586-13598
Issue Date:: 2023-11

Closed Access

	Filename	Description	Size
	DifFormer_Multi-Resolutional_Differencing_Transformer_With_Dynami.pdf	Published version	3.38 MB	Adobe PDF	View/Open

Copyright Clearance Process

Recently Added
In Progress
Closed Access

This item is closed access and not available.

Full metadata record

Field	Value	Language
dc.contributor.author	Li, B
dc.contributor.author	Cui, W
dc.contributor.author	Zhang, L
dc.contributor.author	Zhu, C
dc.contributor.author	Wang, W
dc.contributor.author	Tsang, IW
dc.contributor.author	Zhou, JT
dc.date.accessioned	2024-03-04T00:11:53Z
dc.date.available	2024-03-04T00:11:53Z
dc.date.issued	2023-11
dc.identifier.citation	IEEE Trans Pattern Anal Mach Intell, 2023, 45, (11), pp. 13586-13598
dc.identifier.issn	0162-8828
dc.identifier.issn	1939-3539
dc.identifier.uri	http://hdl.handle.net/10453/176032
dc.description.abstract	Time series analysis is essential to many far-reaching applications of data science and statistics including economic and financial forecasting, surveillance, and automated business processing. Though being greatly successful of Transformer in computer vision and natural language processing, the potential of employing it as the general backbone in analyzing the ubiquitous times series data has not been fully released yet. Prior Transformer variants on time series highly rely on task-dependent designs and pre-assumed "pattern biases", revealing its insufficiency in representing nuanced seasonal, cyclic, and outlier patterns which are highly prevalent in time series. As a consequence, they can not generalize well to different time series analysis tasks. To tackle the challenges, we propose DifFormer, an effective and efficient Transformer architecture that can serve as a workhorse for a variety of time-series analysis tasks. DifFormer incorporates a novel multi-resolutional differencing mechanism, which is able to progressively and adaptively make nuanced yet meaningful changes prominent, meanwhile, the periodic or cyclic patterns can be dynamically captured with flexible lagging and dynamic ranging operations. Extensive experiments demonstrate DifFormer significantly outperforms state-of-the-art models on three essential time-series analysis tasks, including classification, regression, and forecasting. In addition to its superior performances, DifFormer also excels in efficiency - a linear time/memory complexity with empirically lower time consumption.
dc.format	Print-Electronic
dc.language	eng
dc.publisher	IEEE COMPUTER SOC
dc.relation.ispartof	IEEE Trans Pattern Anal Mach Intell
dc.relation.isbasedon	10.1109/TPAMI.2023.3293516
dc.rights	info:eu-repo/semantics/closedAccess
dc.subject	0801 Artificial Intelligence and Image Processing, 0806 Information Systems, 0906 Electrical and Electronic Engineering
dc.subject.classification	Artificial Intelligence & Image Processing
dc.subject.classification	4603 Computer vision and multimedia computation
dc.subject.classification	4611 Machine learning
dc.title	DifFormer: Multi-Resolutional Differencing Transformer With Dynamic Ranging for Time Series Analysis.
dc.type	Journal Article
utslib.citation.volume	45
utslib.location.activity	United States
utslib.for	0801 Artificial Intelligence and Image Processing
utslib.for	0806 Information Systems
utslib.for	0906 Electrical and Electronic Engineering
pubs.organisational-group	University of Technology Sydney
pubs.organisational-group	University of Technology Sydney/Faculty of Engineering and Information Technology
pubs.organisational-group	University of Technology Sydney/Strength - AAII - Australian Artificial Intelligence Institute
utslib.copyright.status	closed_access	*
dc.date.updated	2024-03-04T00:11:50Z
pubs.issue	11
pubs.publication-status	Published
pubs.volume	45
utslib.citation.issue	11

Abstract:

Time series analysis is essential to many far-reaching applications of data science and statistics including economic and financial forecasting, surveillance, and automated business processing. Though being greatly successful of Transformer in computer vision and natural language processing, the potential of employing it as the general backbone in analyzing the ubiquitous times series data has not been fully released yet. Prior Transformer variants on time series highly rely on task-dependent designs and pre-assumed "pattern biases", revealing its insufficiency in representing nuanced seasonal, cyclic, and outlier patterns which are highly prevalent in time series. As a consequence, they can not generalize well to different time series analysis tasks. To tackle the challenges, we propose DifFormer, an effective and efficient Transformer architecture that can serve as a workhorse for a variety of time-series analysis tasks. DifFormer incorporates a novel multi-resolutional differencing mechanism, which is able to progressively and adaptively make nuanced yet meaningful changes prominent, meanwhile, the periodic or cyclic patterns can be dynamically captured with flexible lagging and dynamic ranging operations. Extensive experiments demonstrate DifFormer significantly outperforms state-of-the-art models on three essential time-series analysis tasks, including classification, regression, and forecasting. In addition to its superior performances, DifFormer also excels in efficiency - a linear time/memory complexity with empirically lower time consumption.

Please use this identifier to cite or link to this item:

http://hdl.handle.net/10453/176032