A comparative study of 13 deep reinforcement learning based energy management methods for a hybrid electric vehicle

Wang, H; Ye, Y; Zhang, J; Xu, B

A comparative study of 13 deep reinforcement learning based energy management methods for a hybrid electric vehicle

Wang, H Ye, Y Zhang, J

Xu, B

Permalink

Publisher:: Elsevier
Publication Type:: Journal Article
Citation:: Energy, 2023, 266, pp. 126497
Issue Date:: 2023-03-01

Closed Access

	Filename	Description	Size
	1-s2.0-S0360544222033837-main.pdf	Published version	3.81 MB	Adobe PDF	View/Open

Copyright Clearance Process

Recently Added
In Progress
Closed Access

This item is closed access and not available.

Full metadata record

Field	Value	Language
dc.contributor.author	Wang, H
dc.contributor.author	Ye, Y
dc.contributor.author	Zhang, J https://orcid.org/0000-0003-2616-6722
dc.contributor.author	Xu, B
dc.date.accessioned	2024-03-21T21:41:13Z
dc.date.available	2024-03-21T21:41:13Z
dc.date.issued	2023-03-01
dc.identifier.citation	Energy, 2023, 266, pp. 126497
dc.identifier.issn	0360-5442
dc.identifier.uri	http://hdl.handle.net/10453/176983
dc.description.abstract	Energy management strategy (EMS) has a huge impact on the energy efficiency of hybrid electric vehicles (HEVs). Recently, fast-growing number of studies have applied different deep reinforcement learning (DRL) based EMS for HEVs. However, a unified performance review benchmark is lacking for most popular DRL algorithms. In this study, 13 popular DRL algorithms are applied as HEV EMSs. The reward performance, computation cost, and learning convergence of different DRL algorithms are discussed. In addition, HEV environments are modified to fit both discrete and continuous action spaces. The results show that the stability of agent during the learning process of continuous action space is more stable than discrete action space. In the continuous action space, SAC has the highest reward, and PPO has the lowest time cost. In discrete action space, DQN has the lowest time cost, and FQF has the highest reward. The comparison among SAC, FQF, rule-based, and equivalent consumption minimization strategies (ECMS) shows that DRL EMSs run the engine more efficiently, thus saving fuel consumption. The fuel consumption of FQF is 10.26% and 5.34% less than Rule-based and ECMS, respectively. The contribution of this paper will speed up the application of DRL algorithms in the HEV EMS application.
dc.language	en
dc.publisher	Elsevier
dc.relation.ispartof	Energy
dc.relation.isbasedon	10.1016/j.energy.2022.126497
dc.rights	info:eu-repo/semantics/closedAccess
dc.subject	0913 Mechanical Engineering, 0914 Resources Engineering and Extractive Metallurgy, 0915 Interdisciplinary Engineering
dc.subject.classification	Energy
dc.subject.classification	4008 Electrical engineering
dc.subject.classification	4012 Fluid mechanics and thermal engineering
dc.subject.classification	4017 Mechanical engineering
dc.title	A comparative study of 13 deep reinforcement learning based energy management methods for a hybrid electric vehicle
dc.type	Journal Article
utslib.citation.volume	266
utslib.for	0913 Mechanical Engineering
utslib.for	0914 Resources Engineering and Extractive Metallurgy
utslib.for	0915 Interdisciplinary Engineering
pubs.organisational-group	University of Technology Sydney
pubs.organisational-group	University of Technology Sydney/Faculty of Engineering and Information Technology
pubs.organisational-group	University of Technology Sydney/Faculty of Engineering and Information Technology/School of Electrical and Data Engineering
utslib.copyright.status	closed_access	*
dc.date.updated	2024-03-21T21:41:10Z
pubs.publication-status	Published
pubs.volume	266

Abstract:

Energy management strategy (EMS) has a huge impact on the energy efficiency of hybrid electric vehicles (HEVs). Recently, fast-growing number of studies have applied different deep reinforcement learning (DRL) based EMS for HEVs. However, a unified performance review benchmark is lacking for most popular DRL algorithms. In this study, 13 popular DRL algorithms are applied as HEV EMSs. The reward performance, computation cost, and learning convergence of different DRL algorithms are discussed. In addition, HEV environments are modified to fit both discrete and continuous action spaces. The results show that the stability of agent during the learning process of continuous action space is more stable than discrete action space. In the continuous action space, SAC has the highest reward, and PPO has the lowest time cost. In discrete action space, DQN has the lowest time cost, and FQF has the highest reward. The comparison among SAC, FQF, rule-based, and equivalent consumption minimization strategies (ECMS) shows that DRL EMSs run the engine more efficiently, thus saving fuel consumption. The fuel consumption of FQF is 10.26% and 5.34% less than Rule-based and ECMS, respectively. The contribution of this paper will speed up the application of DRL algorithms in the HEV EMS application.

Please use this identifier to cite or link to this item:

http://hdl.handle.net/10453/176983