Two-Stage Optimization of Virtual Power Plant Operation Considering Substantial Quantity of EVs Participation Using Reinforcement Learning and Gradient-Based Programming

Zhu, R; Qi, J; Wang, J; Li, L

Two-Stage Optimization of Virtual Power Plant Operation Considering Substantial Quantity of EVs Participation Using Reinforcement Learning and Gradient-Based Programming

Zhu, R

Qi, J Wang, J Li, L

Permalink

Publisher:: MDPI
Publication Type:: Journal Article
Citation:: Energies, 2025, 18, (22)
Issue Date:: 2025-11-01

Open Access

Copyright Clearance Process

Recently Added
In Progress
Open Access

This item is open access.

Adobe PDF

Download Published versionAdobe PDF (826.93 kB)

View on publisher's site

View statistics

Full metadata record

Field	Value	Language
dc.contributor.author	Zhu, R https://orcid.org/0009-0005-2395-328X
dc.contributor.author	Qi, J
dc.contributor.author	Wang, J
dc.contributor.author	Li, L https://orcid.org/0000-0001-8485-2216
dc.date.accessioned	2026-01-27T12:19:25Z
dc.date.available	2026-01-27T12:19:25Z
dc.date.issued	2025-11-01
dc.identifier.citation	Energies, 2025, 18, (22)
dc.identifier.issn	1996-1073
dc.identifier.issn	1996-1073
dc.identifier.uri	http://hdl.handle.net/10453/192445
dc.description.abstract	Modern electrical vehicles (EVs) are equipped with sizable batteries that possess significant potential as energy prosumers. EVs are poised to be transformative assets and pivotal contributors to the virtual power plant (VPP), enhancing the performance and profitability of VPPs. The number of household EVs is increasing yearly, and this poses new challenges to the optimization of VPP operations. The computational cost increases exponentially as the number of decision variables rises with the increasing participation of EVs. This paper explores the role of a large number of EVs as prosumers, interacting with a VPP consisting of a photovoltaic system and battery energy storage system. To accommodate the large quantity of EVs in the modeling, this research adopts the decentralized control structure. It optimizes EV operations by regulating their charging and discharging behavior in response to pricing signals from the VPP. A two-stage optimization framework is proposed for VPP-EV operation using a reinforcement algorithm and gradient-based programming. Action masking for reinforcement learning is explored to eliminate invalid actions, reducing ineffective exploration, thereby accelerating the convergence of the algorithm. The proposed approach is capable of handling a substantial number of EVs and addressing the stochastic characteristics of EV charging and discharging behaviors. Simulation results demonstrate that the VPP-EV operation optimization increases the revenue of the VPP and significantly reduces the electricity costs for EV owners. Through the optimization of EV operations, the charging cost of 1000 EVs participating in the V2G services is reduced by 26.38% compared to those that opt out of the scheme, and VPP revenue increases by 27.83% accordingly.
dc.language	English
dc.publisher	MDPI
dc.relation.ispartof	Energies
dc.relation.isbasedon	10.3390/en18225898
dc.rights	info:eu-repo/semantics/openAccess
dc.subject	02 Physical Sciences, 09 Engineering
dc.subject.classification	33 Built environment and design
dc.subject.classification	40 Engineering
dc.subject.classification	51 Physical sciences
dc.title	Two-Stage Optimization of Virtual Power Plant Operation Considering Substantial Quantity of EVs Participation Using Reinforcement Learning and Gradient-Based Programming
dc.type	Journal Article
utslib.citation.volume	18
utslib.for	02 Physical Sciences
utslib.for	09 Engineering
pubs.organisational-group	University of Technology Sydney
pubs.organisational-group	University of Technology Sydney/Faculty of Engineering and Information Technology
pubs.organisational-group	University of Technology Sydney/Faculty of Engineering and Information Technology/School of Electrical and Data Engineering
pubs.organisational-group	University of Technology Sydney/Faculty of Engineering and Information Technology/Engineering and IT Related HDR Students
utslib.copyright.status	open_access	*
dc.rights.license	This work is licensed under a Creative Commons Attribution 4.0 International License (CC BY 4.0). To view a copy of this license, visit https://creativecommons.org/licenses/by/4.0/
dc.date.updated	2026-01-27T12:19:23Z
pubs.issue	22
pubs.publication-status	Published
pubs.volume	18
utslib.citation.issue	22

Abstract:

Modern electrical vehicles (EVs) are equipped with sizable batteries that possess significant potential as energy prosumers. EVs are poised to be transformative assets and pivotal contributors to the virtual power plant (VPP), enhancing the performance and profitability of VPPs. The number of household EVs is increasing yearly, and this poses new challenges to the optimization of VPP operations. The computational cost increases exponentially as the number of decision variables rises with the increasing participation of EVs. This paper explores the role of a large number of EVs as prosumers, interacting with a VPP consisting of a photovoltaic system and battery energy storage system. To accommodate the large quantity of EVs in the modeling, this research adopts the decentralized control structure. It optimizes EV operations by regulating their charging and discharging behavior in response to pricing signals from the VPP. A two-stage optimization framework is proposed for VPP-EV operation using a reinforcement algorithm and gradient-based programming. Action masking for reinforcement learning is explored to eliminate invalid actions, reducing ineffective exploration, thereby accelerating the convergence of the algorithm. The proposed approach is capable of handling a substantial number of EVs and addressing the stochastic characteristics of EV charging and discharging behaviors. Simulation results demonstrate that the VPP-EV operation optimization increases the revenue of the VPP and significantly reduces the electricity costs for EV owners. Through the optimization of EV operations, the charging cost of 1000 EVs participating in the V2G services is reduced by 26.38% compared to those that opt out of the scheme, and VPP revenue increases by 27.83% accordingly.

Please use this identifier to cite or link to this item:

http://hdl.handle.net/10453/192445