The U-turn task is one of the contents of autonomous driving research, and most of the solutions under the standard roads in cities cannot be implemented on non-standard roads. Aiming at solving this problem, this paper established a vehicle U-turn dynamical model and designed a multi-scale convolutional neural network to extract feature maps as the input of the agent. In addition, for the sparse reward problem in the U-turn task, this paper proposed a hierarchical proximal policy optimization algorithm that combined hierarchical reinforcement learning and proximal policy optimization algorithm. In experiments with simple and complex scenarios, this algorithm learns policies faster and has a higher success rate of U-turn compared to other algorithms.

[2022-06-07] Accepted Paper /doc/2022.03.0127/v1

[2022-10-05] Printed Article /doc/2022.03.0127/v2