Leveraging Digital Twin and DRL for Collaborative Context Offloading in C-V2X Autonomous Driving

Sun, K; Wu, J; Pan, Q; Zheng, X; Li, J; Yu, S

Leveraging Digital Twin and DRL for Collaborative Context Offloading in C-V2X Autonomous Driving

Sun, K Wu, J Pan, Q Zheng, X Li, J Yu, S

Permalink

Publisher:: Institute of Electrical and Electronics Engineers (IEEE)
Publication Type:: Journal Article
Citation:: IEEE Transactions on Vehicular Technology, 2023, PP, (99), pp. 1-16
Issue Date:: 2023-01-01

Closed Access

	Filename	Description	Size
	Leveraging_Digital_Twin_and_DRL_for_Collaborative_Context_Offloading_in_C-V2X_Autonomous_Driving.pdf	Accepted version	43.09 MB	Adobe PDF	View/Open

Copyright Clearance Process

Recently Added
In Progress
Closed Access

This item is closed access and not available.

Full metadata record

Field	Value	Language
dc.contributor.author	Sun, K
dc.contributor.author	Wu, J
dc.contributor.author	Pan, Q
dc.contributor.author	Zheng, X
dc.contributor.author	Li, J
dc.contributor.author	Yu, S https://orcid.org/0000-0003-4485-6743
dc.date.accessioned	2024-04-03T03:26:22Z
dc.date.available	2024-04-03T03:26:22Z
dc.date.issued	2023-01-01
dc.identifier.citation	IEEE Transactions on Vehicular Technology, 2023, PP, (99), pp. 1-16
dc.identifier.issn	0018-9545
dc.identifier.issn	1939-9359
dc.identifier.uri	http://hdl.handle.net/10453/177460
dc.description.abstract	Digital Twin (DT) technology, as a promising technology, can achieve the vehicular contexts mapping of the virtual world and physical world in a collaborative autonomous driving (CAD) system. DT technology is developed on the basis of C-V2X, 6G, Mobile Edge Computing (MEC), Machine Learning (ML) and other technologies, which can enable the creation of robust and reliable digital twin-based collaborative autonomous driving architectures, providing a platform for testing, validating, and refining autonomous driving systems in a highly efficient and safe manner. However, the future large-scale CAD system needs greater real-time processing and resource collaboration capability for autonomous vehicles (AVs). Especially considering the mobility of AVs, it puts higher demands on the management of AVs. In this paper, we present a digital twin (DT)-based collaborative autonomous driving (DTCAD) three-layer architecture in C-V2X to provide better resource management of AVs. In order to improve the Quality of Service (QoS) and reduce the processing latency in large-scale CAD scenarios, a scalable Deep Reinforcement Learning and Mean Field Game method (DDPG-MFG) are proposed, where the dynamic and real-time interaction between AVs is approximated as a mean-field gaming process in DT resource allocation. Especially, to improve the interaction efficiency between AVs and CAD environment, we design more efficient exploitation and exploration algorithms for AVs. The CARLA simulation demonstrates our proposed algorithm significantly reduces the task offloading latency, and improves the average rewards by 28.5%, 3.5%, and 6.8%, compared with traditional DDPG, TD3, and AC, respectively.
dc.language	en
dc.publisher	Institute of Electrical and Electronics Engineers (IEEE)
dc.relation.ispartof	IEEE Transactions on Vehicular Technology
dc.relation.isbasedon	10.1109/TVT.2023.3333243
dc.rights	info:eu-repo/semantics/closedAccess
dc.subject	08 Information and Computing Sciences, 09 Engineering, 10 Technology
dc.subject.classification	Automobile Design & Engineering
dc.subject.classification	40 Engineering
dc.subject.classification	46 Information and computing sciences
dc.title	Leveraging Digital Twin and DRL for Collaborative Context Offloading in C-V2X Autonomous Driving
dc.type	Journal Article
utslib.citation.volume	PP
utslib.for	08 Information and Computing Sciences
utslib.for	09 Engineering
utslib.for	10 Technology
pubs.organisational-group	University of Technology Sydney
pubs.organisational-group	University of Technology Sydney/Faculty of Engineering and Information Technology
pubs.organisational-group	University of Technology Sydney/Faculty of Engineering and Information Technology/School of Computer Science
pubs.organisational-group	University of Technology Sydney/Strength - CCSP - Centre for Cyber Security and Privacy
utslib.copyright.status	closed_access	*
dc.date.updated	2024-04-03T03:26:01Z
pubs.issue	99
pubs.publication-status	Published
pubs.volume	PP
utslib.citation.issue	99

Abstract:

Digital Twin (DT) technology, as a promising technology, can achieve the vehicular contexts mapping of the virtual world and physical world in a collaborative autonomous driving (CAD) system. DT technology is developed on the basis of C-V2X, 6G, Mobile Edge Computing (MEC), Machine Learning (ML) and other technologies, which can enable the creation of robust and reliable digital twin-based collaborative autonomous driving architectures, providing a platform for testing, validating, and refining autonomous driving systems in a highly efficient and safe manner. However, the future large-scale CAD system needs greater real-time processing and resource collaboration capability for autonomous vehicles (AVs). Especially considering the mobility of AVs, it puts higher demands on the management of AVs. In this paper, we present a digital twin (DT)-based collaborative autonomous driving (DTCAD) three-layer architecture in C-V2X to provide better resource management of AVs. In order to improve the Quality of Service (QoS) and reduce the processing latency in large-scale CAD scenarios, a scalable Deep Reinforcement Learning and Mean Field Game method (DDPG-MFG) are proposed, where the dynamic and real-time interaction between AVs is approximated as a mean-field gaming process in DT resource allocation. Especially, to improve the interaction efficiency between AVs and CAD environment, we design more efficient exploitation and exploration algorithms for AVs. The CARLA simulation demonstrates our proposed algorithm significantly reduces the task offloading latency, and improves the average rewards by 28.5%, 3.5%, and 6.8%, compared with traditional DDPG, TD3, and AC, respectively.

Please use this identifier to cite or link to this item:

http://hdl.handle.net/10453/177460