DeepPlace: Deep reinforcement learning for adaptive flow rule placement in Software-Defined IoT Networks

Nguyen, TG; Phan, TV; Hoang, DT; Nguyen, HH; Le, DT

DeepPlace: Deep reinforcement learning for adaptive flow rule placement in Software-Defined IoT Networks

Nguyen, TG Phan, TV Hoang, DT Nguyen, HH Le, DT

Permalink

Publisher:: ELSEVIER
Publication Type:: Journal Article
Citation:: Computer Communications, 2022, 181, pp. 156-163
Issue Date:: 2022-01-01

Closed Access

	Filename	Description	Size
	1-s2.0-S0140366421003789-main.pdf	Published version	1.02 MB	Adobe PDF	View/Open

Copyright Clearance Process

Recently Added
In Progress
Closed Access

This item is closed access and not available.

Full metadata record

Field	Value	Language
dc.contributor.author	Nguyen, TG
dc.contributor.author	Phan, TV
dc.contributor.author	Hoang, DT
dc.contributor.author	Nguyen, HH
dc.contributor.author	Le, DT
dc.date.accessioned	2022-03-27T00:56:18Z
dc.date.available	2022-03-27T00:56:18Z
dc.date.issued	2022-01-01
dc.identifier.citation	Computer Communications, 2022, 181, pp. 156-163
dc.identifier.issn	0140-3664
dc.identifier.issn	1873-703X
dc.identifier.uri	http://hdl.handle.net/10453/155595
dc.description.abstract	In this paper, we propose a novel and adaptive flow rule placement system based on deep reinforcement learning, namely DeepPlace, in Software-Defined Internet of Things (SDIoT) networks. DeepPlace can provide a fine-grained traffic analysis capability while assuring QoS of traffic flows and proactively avoiding the flow-table overflow issue in the data plane. Specifically, we first investigate the traffic forwarding process in an SDIoT network, i.e., routing and flow rule placement tasks. We design a cost function for the routing to set up traffic flow paths in the data plane. Next, we propose an adaptive flow rule placement approach to maximize the number of match-fields in a flow rule at SDN switches. To deal with the dynamics of IoT traffic flows, we model the system operation by using the Markov decision process (MDP) with a continuous action space and formulate its optimization problem. Subsequently, we develop a deep deterministic policy gradient-based algorithm to help the system obtain the optimal policy. The evaluation results demonstrate that DeepPlace can efficiently maintain a significant number of match-fields in a flow rule, i.e., approximately 86% of the maximum level, while minimizing the QoS violation ratio of traffic flows, i.e., 6.7%, in a highly dynamic traffic scenario, which outperforms three other existing solutions, i.e., FlowMan, FlowStat, and DeepMatch.
dc.language	English
dc.publisher	ELSEVIER
dc.relation.ispartof	Computer Communications
dc.relation.isbasedon	10.1016/j.comcom.2021.10.006
dc.rights	info:eu-repo/semantics/closedAccess
dc.subject	0805 Distributed Computing, 0906 Electrical and Electronic Engineering, 1005 Communications Technologies
dc.subject.classification	Networking & Telecommunications
dc.title	DeepPlace: Deep reinforcement learning for adaptive flow rule placement in Software-Defined IoT Networks
dc.type	Journal Article
utslib.citation.volume	181
utslib.for	0805 Distributed Computing
utslib.for	0906 Electrical and Electronic Engineering
utslib.for	1005 Communications Technologies
pubs.organisational-group	/University of Technology Sydney
pubs.organisational-group	/University of Technology Sydney/Faculty of Engineering and Information Technology
pubs.organisational-group	/University of Technology Sydney/Strength - GBDTC - Global Big Data Technologies
pubs.organisational-group	/University of Technology Sydney/Faculty of Engineering and Information Technology/School of Electrical and Data Engineering
utslib.copyright.status	closed_access	*
dc.date.updated	2022-03-27T00:56:17Z
pubs.publication-status	Published
pubs.volume	181

Abstract:

In this paper, we propose a novel and adaptive flow rule placement system based on deep reinforcement learning, namely DeepPlace, in Software-Defined Internet of Things (SDIoT) networks. DeepPlace can provide a fine-grained traffic analysis capability while assuring QoS of traffic flows and proactively avoiding the flow-table overflow issue in the data plane. Specifically, we first investigate the traffic forwarding process in an SDIoT network, i.e., routing and flow rule placement tasks. We design a cost function for the routing to set up traffic flow paths in the data plane. Next, we propose an adaptive flow rule placement approach to maximize the number of match-fields in a flow rule at SDN switches. To deal with the dynamics of IoT traffic flows, we model the system operation by using the Markov decision process (MDP) with a continuous action space and formulate its optimization problem. Subsequently, we develop a deep deterministic policy gradient-based algorithm to help the system obtain the optimal policy. The evaluation results demonstrate that DeepPlace can efficiently maintain a significant number of match-fields in a flow rule, i.e., approximately 86% of the maximum level, while minimizing the QoS violation ratio of traffic flows, i.e., 6.7%, in a highly dynamic traffic scenario, which outperforms three other existing solutions, i.e., FlowMan, FlowStat, and DeepMatch.

Please use this identifier to cite or link to this item:

http://hdl.handle.net/10453/155595