Augmented Deep Reinforcement Learning for Online Energy Minimization of Wireless Powered Mobile Edge Computing

Chen, X; Dai, W; Ni, W; Wang, X; Zhang, S; Xu, S; Sun, Y

Augmented Deep Reinforcement Learning for Online Energy Minimization of Wireless Powered Mobile Edge Computing

Chen, X Dai, W Ni, W

Wang, X Zhang, S Xu, S Sun, Y

Permalink

Publisher:: IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
Publication Type:: Journal Article
Citation:: IEEE Transactions on Communications, 2023, 71, (5), pp. 2698-2710
Issue Date:: 2023-05-01

Closed Access

	Filename	Description	Size
	Augmented_Deep_Reinforcement_Learning_for_Online_Energy_Minimization_of_Wireless_Powered_Mobile_Edge_Computing.pdf	Published version	16.96 MB	Adobe PDF	View/Open

Copyright Clearance Process

Recently Added
In Progress
Closed Access

This item is closed access and not available.

Full metadata record

Field	Value	Language
dc.contributor.author	Chen, X
dc.contributor.author	Dai, W
dc.contributor.author	Ni, W https://orcid.org/0000-0002-4933-594X
dc.contributor.author	Wang, X
dc.contributor.author	Zhang, S
dc.contributor.author	Xu, S
dc.contributor.author	Sun, Y
dc.date.accessioned	2024-04-04T23:08:31Z
dc.date.available	2024-04-04T23:08:31Z
dc.date.issued	2023-05-01
dc.identifier.citation	IEEE Transactions on Communications, 2023, 71, (5), pp. 2698-2710
dc.identifier.issn	0090-6778
dc.identifier.issn	1558-0857
dc.identifier.uri	http://hdl.handle.net/10453/177479
dc.description.abstract	Mobile edge computing (MEC) offers an opportunity for devices relying on wireless power transfer (WPT), to accomplish computationally demanding tasks. Such WPT-powered MEC systems have yet to be optimized for long-term efficiency, due to random and changing task demands and wireless channel states of the devices. This paper presents an augmented two-staged deep Q-network (DQN), referred to as 'TS-DQN,' for online optimization of WPT-powered MEC systems, where the WPT, offloading schedule, channel allocation, and the CPU configurations of the edge server and devices are jointly optimized to minimize the long-term average energy requirement of the systems. The key idea is to design a DQN for learning the channel allocation and task admission, while the WPT, offloading time and CPU configurations are efficiently optimized to precisely evaluate the reward of the DQN and substantially reduce its action space. Another important aspect is that a new action generation method is developed to expand and diversify the actions of the DQN, further accelerating its convergence. As validated by simulations, the proposed TS-DQN is much more energy efficient and converges much faster, than its potential alternative directly using the state-of-the-art Deep Deterministic Policy Gradient algorithm to learn all decision variables.
dc.language	English
dc.publisher	IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
dc.relation.ispartof	IEEE Transactions on Communications
dc.relation.isbasedon	10.1109/TCOMM.2023.3251353
dc.rights	info:eu-repo/semantics/closedAccess
dc.subject	0804 Data Format, 0906 Electrical and Electronic Engineering, 1005 Communications Technologies
dc.subject.classification	4006 Communications engineering
dc.subject.classification	4009 Electronics, sensors and digital hardware
dc.subject.classification	4606 Distributed computing and systems software
dc.title	Augmented Deep Reinforcement Learning for Online Energy Minimization of Wireless Powered Mobile Edge Computing
dc.type	Journal Article
utslib.citation.volume	71
utslib.for	0804 Data Format
utslib.for	0906 Electrical and Electronic Engineering
utslib.for	1005 Communications Technologies
pubs.organisational-group	University of Technology Sydney
pubs.organisational-group	University of Technology Sydney/Faculty of Engineering and Information Technology
pubs.organisational-group	University of Technology Sydney/Faculty of Engineering and Information Technology/School of Electrical and Data Engineering
utslib.copyright.status	closed_access	*
dc.date.updated	2024-04-04T23:08:28Z
pubs.issue	5
pubs.publication-status	Published
pubs.volume	71
utslib.citation.issue	5

Abstract:

Mobile edge computing (MEC) offers an opportunity for devices relying on wireless power transfer (WPT), to accomplish computationally demanding tasks. Such WPT-powered MEC systems have yet to be optimized for long-term efficiency, due to random and changing task demands and wireless channel states of the devices. This paper presents an augmented two-staged deep Q-network (DQN), referred to as 'TS-DQN,' for online optimization of WPT-powered MEC systems, where the WPT, offloading schedule, channel allocation, and the CPU configurations of the edge server and devices are jointly optimized to minimize the long-term average energy requirement of the systems. The key idea is to design a DQN for learning the channel allocation and task admission, while the WPT, offloading time and CPU configurations are efficiently optimized to precisely evaluate the reward of the DQN and substantially reduce its action space. Another important aspect is that a new action generation method is developed to expand and diversify the actions of the DQN, further accelerating its convergence. As validated by simulations, the proposed TS-DQN is much more energy efficient and converges much faster, than its potential alternative directly using the state-of-the-art Deep Deterministic Policy Gradient algorithm to learn all decision variables.

Please use this identifier to cite or link to this item:

http://hdl.handle.net/10453/177479