Enhancing Car-Following Performance in Traffic Oscillations Using Expert Demonstration Reinforcement Learning

Li, M; Li, Z; Cao, Z

Enhancing Car-Following Performance in Traffic Oscillations Using Expert Demonstration Reinforcement Learning

Li, M Li, Z Cao, Z

Permalink

Publisher:: IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
Publication Type:: Journal Article
Citation:: IEEE Transactions on Intelligent Transportation Systems, 2024, 25, (7), pp. 7751-7766
Issue Date:: 2024-01-01

Closed Access

	Filename	Description	Size
	1717222.pdf	Published version	2.95 MB	Adobe PDF	View/Open

Copyright Clearance Process

Recently Added
In Progress
Closed Access

This item is closed access and not available.

Full metadata record

Field	Value	Language
dc.contributor.author	Li, M
dc.contributor.author	Li, Z
dc.contributor.author	Cao, Z https://orcid.org/0000-0003-3656-0328
dc.date.accessioned	2024-08-07T05:12:42Z
dc.date.available	2024-08-07T05:12:42Z
dc.date.issued	2024-01-01
dc.identifier.citation	IEEE Transactions on Intelligent Transportation Systems, 2024, 25, (7), pp. 7751-7766
dc.identifier.issn	1524-9050
dc.identifier.issn	1558-0016
dc.identifier.uri	http://hdl.handle.net/10453/180283
dc.description.abstract	Deep reinforcement learning (DRL) algorithms often face challenges in achieving stability and efficiency due to significant policy gradient variance and inaccurate reward function estimation in complex scenarios. This study addresses these issues in the context of multi-objective car-following control tasks with time lag in traffic oscillations. We propose an expert demonstration reinforcement learning (EDRL) approach that aims to stabilize training, accelerate learning, and enhance car-following performance. The key idea is to leverage expert demonstrations, which represent superior car-following control experiences, to improve the DRL policy. Our method involves two sequential steps. In the first step, expert demonstrations are obtained during offline pretraining by utilizing prior traffic knowledge, including car-following trajectories from an empirical database and classic car-following models. In the second step, expert demonstrations are obtained during online training, where the agent interacts with the car-following environment. The EDRL agents are trained through supervised regression on the expert demonstrations using the behavioral cloning technique. Experimental results conducted in various traffic oscillation scenarios demonstrate that our proposed method significantly enhances training stability, learning speed, and rewards compared to baseline algorithms.
dc.language	English
dc.publisher	IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
dc.relation.ispartof	IEEE Transactions on Intelligent Transportation Systems
dc.relation.isbasedon	10.1109/TITS.2024.3368474
dc.rights	info:eu-repo/semantics/closedAccess
dc.subject	0801 Artificial Intelligence and Image Processing, 0905 Civil Engineering, 1507 Transportation and Freight Services
dc.subject.classification	Logistics & Transportation
dc.subject.classification	3509 Transportation, logistics and supply chains
dc.subject.classification	4602 Artificial intelligence
dc.subject.classification	4603 Computer vision and multimedia computation
dc.title	Enhancing Car-Following Performance in Traffic Oscillations Using Expert Demonstration Reinforcement Learning
dc.type	Journal Article
utslib.citation.volume	25
utslib.for	0801 Artificial Intelligence and Image Processing
utslib.for	0905 Civil Engineering
utslib.for	1507 Transportation and Freight Services
pubs.organisational-group	University of Technology Sydney
pubs.organisational-group	University of Technology Sydney/Faculty of Engineering and Information Technology
pubs.organisational-group	University of Technology Sydney/Faculty of Engineering and Information Technology/School of Computer Science
utslib.copyright.status	closed_access	*
dc.date.updated	2024-08-07T05:12:38Z
pubs.issue	7
pubs.publication-status	Published
pubs.volume	25
utslib.citation.issue	7

Abstract:

Deep reinforcement learning (DRL) algorithms often face challenges in achieving stability and efficiency due to significant policy gradient variance and inaccurate reward function estimation in complex scenarios. This study addresses these issues in the context of multi-objective car-following control tasks with time lag in traffic oscillations. We propose an expert demonstration reinforcement learning (EDRL) approach that aims to stabilize training, accelerate learning, and enhance car-following performance. The key idea is to leverage expert demonstrations, which represent superior car-following control experiences, to improve the DRL policy. Our method involves two sequential steps. In the first step, expert demonstrations are obtained during offline pretraining by utilizing prior traffic knowledge, including car-following trajectories from an empirical database and classic car-following models. In the second step, expert demonstrations are obtained during online training, where the agent interacts with the car-following environment. The EDRL agents are trained through supervised regression on the expert demonstrations using the behavioral cloning technique. Experimental results conducted in various traffic oscillation scenarios demonstrate that our proposed method significantly enhances training stability, learning speed, and rewards compared to baseline algorithms.

Please use this identifier to cite or link to this item:

http://hdl.handle.net/10453/180283