Decoupled Progressive Distillation for Sequential Prediction with Interaction Dynamics

Hu, K; Li, L; Xie, Q; Liu, J; Tao, X; Xu, G

Decoupled Progressive Distillation for Sequential Prediction with Interaction Dynamics

Hu, K Li, L Xie, Q Liu, J Tao, X Xu, G

Permalink

Publisher:: Association for Computing Machinery (ACM)
Publication Type:: Journal Article
Citation:: ACM Transactions on Information Systems, 2024, 42, (3), pp. 1-35
Issue Date:: 2024-05-31

Open Access

Copyright Clearance Process

Recently Added
In Progress
Open Access

This item is open access.

Adobe PDF

Download Accepted versionAdobe PDF (2.03 MB)

View on publisher's site

View statistics

Full metadata record

Field	Value	Language
dc.contributor.author	Hu, K
dc.contributor.author	Li, L
dc.contributor.author	Xie, Q
dc.contributor.author	Liu, J
dc.contributor.author	Tao, X
dc.contributor.author	Xu, G https://orcid.org/0000-0003-4493-6663
dc.date.accessioned	2024-06-05T01:46:27Z
dc.date.available	2024-06-05T01:46:27Z
dc.date.issued	2024-05-31
dc.identifier.citation	ACM Transactions on Information Systems, 2024, 42, (3), pp. 1-35
dc.identifier.issn	1046-8188
dc.identifier.issn	1558-2868
dc.identifier.uri	http://hdl.handle.net/10453/179401
dc.description.abstract	<jats:p>Sequential prediction has great value for resource allocation due to its capability in analyzing intents for next prediction. A fundamental challenge arises from real-world interaction dynamics where similar sequences involving multiple intents may exhibit different next items. More importantly, the character of volume candidate items in sequential prediction may amplify such dynamics, making deep networks hard to capture comprehensive intents. This article presents a sequential prediction framework with Decoupled Progressive Distillation (DePoD), drawing on the progressive nature of human cognition. We redefine target and non-target item distillation according to their different effects in the decoupled formulation. This can be achieved through two aspects: (1) Regarding how to learn, our target item distillation with progressive difficulty increases the contribution of low-confidence samples in the later training phase while keeping high-confidence samples in the earlier phase. And, the non-target item distillation starts from a small subset of non-target items from which size increases according to the item frequency. (2) Regarding whom to learn from, a difference evaluator is utilized to progressively select an expert that provides informative knowledge among items from the cohort of peers. Extensive experiments on four public datasets show DePoD outperforms state-of-the-art methods in terms of accuracy-based metrics.</jats:p>
dc.language	en
dc.publisher	Association for Computing Machinery (ACM)
dc.relation.ispartof	ACM Transactions on Information Systems
dc.relation.isbasedon	10.1145/3632403
dc.rights	info:eu-repo/semantics/openAccess
dc.rights	© 2023. This is the author's version of the work. It is posted here for your personal use. Not for redistribution. The definitive Version of Record was published in ACM Transactions on Information Systems, Volume 42 Issue 3 Article No.: 72 pp 1–35 https://dx.doi.org/10.1145/10.1145/3632403.
dc.subject	0806 Information Systems, 0807 Library and Information Studies
dc.subject.classification	Information Systems
dc.subject.classification	4605 Data management and data science
dc.title	Decoupled Progressive Distillation for Sequential Prediction with Interaction Dynamics
dc.type	Journal Article
utslib.citation.volume	42
utslib.for	0806 Information Systems
utslib.for	0807 Library and Information Studies
pubs.organisational-group	University of Technology Sydney
pubs.organisational-group	University of Technology Sydney/Faculty of Engineering and Information Technology
pubs.organisational-group	University of Technology Sydney/Strength - AAI - Advanced Analytics Institute Research Centre
pubs.organisational-group	University of Technology Sydney/Faculty of Engineering and Information Technology/School of Computer Science
utslib.copyright.status	open_access	*
dc.date.updated	2024-06-05T01:46:21Z
pubs.issue	3
pubs.publication-status	Published online
pubs.volume	42
utslib.citation.issue	3

Abstract:

Sequential prediction has great value for resource allocation due to its capability in analyzing intents for next prediction. A fundamental challenge arises from real-world interaction dynamics where similar sequences involving multiple intents may exhibit different next items. More importantly, the character of volume candidate items in sequential prediction may amplify such dynamics, making deep networks hard to capture comprehensive intents. This article presents a sequential prediction framework with Decoupled Progressive Distillation (DePoD), drawing on the progressive nature of human cognition. We redefine target and non-target item distillation according to their different effects in the decoupled formulation. This can be achieved through two aspects: (1) Regarding how to learn, our target item distillation with progressive difficulty increases the contribution of low-confidence samples in the later training phase while keeping high-confidence samples in the earlier phase. And, the non-target item distillation starts from a small subset of non-target items from which size increases according to the item frequency. (2) Regarding whom to learn from, a difference evaluator is utilized to progressively select an expert that provides informative knowledge among items from the cohort of peers. Extensive experiments on four public datasets show DePoD outperforms state-of-the-art methods in terms of accuracy-based metrics.

Please use this identifier to cite or link to this item:

http://hdl.handle.net/10453/179401