MTPChat: A Multimodal Time-Aware Persona Dataset for Conversational Agents

Yang, W; Li, Y; Fang, M; Chen, L

MTPChat: A Multimodal Time-Aware Persona Dataset for Conversational Agents

Yang, W

Li, Y

Fang, M Chen, L

Permalink

Publisher:: Association for Computational Linguistics (ACL)
Publication Type:: Conference Proceeding
Citation:: 2025 Annual Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics Proceedings of the Conference Findings Naacl 2025, 2025, pp. 5830-5841
Issue Date:: 2025-01-01

Open Access

Copyright Clearance Process

Recently Added
In Progress
Open Access

This item is open access.

Adobe PDF

Download Published versionAdobe PDF (2.1 MB)

View on publisher's site

View statistics

Full metadata record

Field	Value	Language
dc.contributor.author	Yang, W https://orcid.org/0000-0002-6449-288X
dc.contributor.author	Li, Y https://orcid.org/0009-0007-1123-9673
dc.contributor.author	Fang, M
dc.contributor.author	Chen, L https://orcid.org/0000-0002-6468-5729
dc.date	2025-04
dc.date.accessioned	2026-06-02T05:29:00Z
dc.date.available	2026-06-02T05:29:00Z
dc.date.issued	2025-01-01
dc.identifier.citation	2025 Annual Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics Proceedings of the Conference Findings Naacl 2025, 2025, pp. 5830-5841
dc.identifier.uri	http://hdl.handle.net/10453/195209
dc.description.abstract	Understanding temporal dynamics is critical for conversational agents, enabling effective content analysis and informed decision-making. However, time-aware datasets, particularly for persona-grounded conversations, are still limited, which narrows their scope and diminishes their complexity. To address this gap, we introduce MTPChat, a multimodal, time-aware persona dialogue dataset that integrates linguistic, visual, and temporal elements within dialogue and persona memory. Leveraging MTPChat, we propose two time-sensitive tasks: Temporal Next Response Prediction (TNRP) and Temporal Grounding Memory Prediction (TGMP), both designed to assess a model’s ability to understand implicit temporal cues and dynamic interactions. Additionally, we present an innovative framework featuring an adaptive temporal module to effectively integrate multimodal streams and capture temporal dependencies. Experimental results validate the challenges posed by MTPChat and demonstrate the effectiveness of our framework in multimodal time-sensitive scenarios.
dc.language	en
dc.publisher	Association for Computational Linguistics (ACL)
dc.relation.ispartof	2025 Annual Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics Proceedings of the Conference Findings Naacl 2025
dc.relation.ispartof	Findings of the Association for Computational Linguistics: NAACL 2025
dc.relation.isbasedon	10.18653/v1/2025.findings-naacl.323
dc.rights	info:eu-repo/semantics/openAccess
dc.title	MTPChat: A Multimodal Time-Aware Persona Dataset for Conversational Agents
dc.type	Conference Proceeding
pubs.organisational-group	University of Technology Sydney
pubs.organisational-group	University of Technology Sydney/Faculty of Engineering and Information Technology
pubs.organisational-group	University of Technology Sydney/Faculty of Engineering and Information Technology/School of Computer Science
pubs.organisational-group	University of Technology Sydney/UTS Groups
pubs.organisational-group	University of Technology Sydney/UTS Groups/Australian Artificial Intelligence Institute (AAII)
pubs.organisational-group	University of Technology Sydney/UTS Groups/The Trustworthy Digital Society
pubs.organisational-group	University of Technology Sydney/Faculty of Engineering and Information Technology/Engineering and IT Related HDR Students
utslib.copyright.status	open_access	*
dc.rights.license	This work is licensed under a Creative Commons Attribution 4.0 International License (CC BY 4.0). To view a copy of this license, visit https://creativecommons.org/licenses/by/4.0/
dc.date.updated	2026-06-02T05:28:59Z
pubs.finish-date	2025-04
pubs.publication-status	Published
pubs.start-date	2025-04

Abstract:

Understanding temporal dynamics is critical for conversational agents, enabling effective content analysis and informed decision-making. However, time-aware datasets, particularly for persona-grounded conversations, are still limited, which narrows their scope and diminishes their complexity. To address this gap, we introduce MTPChat, a multimodal, time-aware persona dialogue dataset that integrates linguistic, visual, and temporal elements within dialogue and persona memory. Leveraging MTPChat, we propose two time-sensitive tasks: Temporal Next Response Prediction (TNRP) and Temporal Grounding Memory Prediction (TGMP), both designed to assess a model’s ability to understand implicit temporal cues and dynamic interactions. Additionally, we present an innovative framework featuring an adaptive temporal module to effectively integrate multimodal streams and capture temporal dependencies. Experimental results validate the challenges posed by MTPChat and demonstrate the effectiveness of our framework in multimodal time-sensitive scenarios.

Please use this identifier to cite or link to this item:

http://hdl.handle.net/10453/195209