Preceding vehicle following algorithm with human driving characteristics

作者单位：College of Information Science & Technology, Beijing University of Chemical Technology；Beijing Key Laboratory of Information Service Engineering, Beijing Union University

发表期刊：Automobile Engineering 2021, Vol. 235(7)

摘要：This paper proposes a new approach of using reinforcement learning (RL) to train an agent to perform the task of vehicle following with human driving characteristics. We refer to the ideal of inverse reinforcement learning to design thereward function of the RL model. The factors that need to be weighed in vehicle following were vectorized into reward vectors, and the reward function was defined as the inner product of the reward vector and weights. Driving data of human drivers was collected and analyzed to obtain the true reward function. The RL model was trained with the deterministic policy gradient algorithm because the state and action spaces are continuous. We adjusted the weight vector of the reward function so that the value vector of the RL model could continuously approach that of a human driver. After dozens of rounds of training, we selected the policy with the nearest value vector to that of a human driver and tested it in the PanoSim simulation environment. The results showed the desired performance for the task of an agent following the preceding vehicle safely and smoothly.

关键词：Inverse reinforcement learning, reinforcement learning, human driver, vehicle, reward vector