Value-based reinforcement learning approaches for task offloading in Delay Constrained Vehicular Edge Computing

Son, DB; Binh, TH; Vo, HK; Nguyen, BM; Binh, HTT; Yu, S

Value-based reinforcement learning approaches for task offloading in Delay Constrained Vehicular Edge Computing

Son, DB Binh, TH Vo, HK Nguyen, BM Binh, HTT Yu, S

Permalink

Publisher:: PERGAMON-ELSEVIER SCIENCE LTD
Publication Type:: Journal Article
Citation:: Engineering Applications of Artificial Intelligence, 2022, 113
Issue Date:: 2022-08-01

Closed Access

	Filename	Description	Size
	1-s2.0-S0952197622001336-main.pdf	Published version	994.51 kB	Adobe PDF	View/Open

Copyright Clearance Process

Recently Added
In Progress
Closed Access

This item is closed access and not available.

Full metadata record

Field	Value	Language
dc.contributor.author	Son, DB
dc.contributor.author	Binh, TH
dc.contributor.author	Vo, HK
dc.contributor.author	Nguyen, BM
dc.contributor.author	Binh, HTT
dc.contributor.author	Yu, S https://orcid.org/0000-0003-4485-6743
dc.date.accessioned	2022-12-07T20:09:24Z
dc.date.available	2022-12-07T20:09:24Z
dc.date.issued	2022-08-01
dc.identifier.citation	Engineering Applications of Artificial Intelligence, 2022, 113
dc.identifier.issn	0952-1976
dc.identifier.issn	1873-6769
dc.identifier.uri	http://hdl.handle.net/10453/164212
dc.description.abstract	In the age of booming information technology, human-being has witnessed the need for new paradigms with both high computational capability and low latency. A potential solution is Vehicular Edge Computing (VEC). Previous work proposed a Fuzzy Deep Q-Network in Offloading scheme (FDQO) that combines Fuzzy rules and Deep Q-Network (DQN) to improve DQN's early performance by using Fuzzy Controller (FC). However, we notice that frequent usage of FC can hinder the future growth performance of model. One way to overcome this issue is to remove Fuzzy Controller entirely. We introduced an algorithm called baseline DQN (b-DQN), represented by its two variants Static baseline DQN (Sb-DQN) and Dynamic baseline DQN (Db-DQN), to modify the exploration rate base on the average rewards of closest observations. Our findings confirm that these baseline DQN algorithms surpass traditional DQN models in terms of average Quality of Experience (QoE) in 100 time slots by about 6%, but still suffer from poor early performance (such as in the first 5 time slots). Here, we introduce baseline FDQO (b-FDQO). This algorithm has a strategy to modify the Fuzzy Logic usage instead of removing it entirely while still observing the rewards to modify the exploration rate. It brings a higher average QoE in the first 5 time slots compared to other non-fuzzy-logic algorithms by at least 55.12%, prevent the model from getting too bad result over all time slots, while having the late performance as good as that of b-DQN.
dc.language	English
dc.publisher	PERGAMON-ELSEVIER SCIENCE LTD
dc.relation.ispartof	Engineering Applications of Artificial Intelligence
dc.relation.isbasedon	10.1016/j.engappai.2022.104898
dc.rights	info:eu-repo/semantics/closedAccess
dc.subject	08 Information and Computing Sciences, 09 Engineering
dc.subject.classification	Artificial Intelligence & Image Processing
dc.title	Value-based reinforcement learning approaches for task offloading in Delay Constrained Vehicular Edge Computing
dc.type	Journal Article
utslib.citation.volume	113
utslib.for	08 Information and Computing Sciences
utslib.for	09 Engineering
pubs.organisational-group	/University of Technology Sydney
pubs.organisational-group	/University of Technology Sydney/Faculty of Engineering and Information Technology
pubs.organisational-group	/University of Technology Sydney/Faculty of Engineering and Information Technology/School of Computer Science
utslib.copyright.status	closed_access	*
dc.date.updated	2022-12-07T20:09:16Z
pubs.publication-status	Published
pubs.volume	113

Abstract:

In the age of booming information technology, human-being has witnessed the need for new paradigms with both high computational capability and low latency. A potential solution is Vehicular Edge Computing (VEC). Previous work proposed a Fuzzy Deep Q-Network in Offloading scheme (FDQO) that combines Fuzzy rules and Deep Q-Network (DQN) to improve DQN's early performance by using Fuzzy Controller (FC). However, we notice that frequent usage of FC can hinder the future growth performance of model. One way to overcome this issue is to remove Fuzzy Controller entirely. We introduced an algorithm called baseline DQN (b-DQN), represented by its two variants Static baseline DQN (Sb-DQN) and Dynamic baseline DQN (Db-DQN), to modify the exploration rate base on the average rewards of closest observations. Our findings confirm that these baseline DQN algorithms surpass traditional DQN models in terms of average Quality of Experience (QoE) in 100 time slots by about 6%, but still suffer from poor early performance (such as in the first 5 time slots). Here, we introduce baseline FDQO (b-FDQO). This algorithm has a strategy to modify the Fuzzy Logic usage instead of removing it entirely while still observing the rewards to modify the exploration rate. It brings a higher average QoE in the first 5 time slots compared to other non-fuzzy-logic algorithms by at least 55.12%, prevent the model from getting too bad result over all time slots, while having the late performance as good as that of b-DQN.

Please use this identifier to cite or link to this item:

http://hdl.handle.net/10453/164212