Aiming at the problem of dimension explosion and sparse rewards caused by the increase in the number of agents and the dynamic instability of the environment in most multi-agent reinforcement learning algorithms, this paper proposed a multi-agent hierarchical reinforcement learning skill discovery algorithm based on weighted value function decomposition. Firstly, the algorithm combined the architecture of centralized training and decentralized execution with hierarchical reinforcement learning, and adopted the method of weighted value function decomposition in the upper level to solve the problem that agents tended to ignore the optimal strategy and chose the suboptimal strategy in the training process. Secondly, it adopted the independent Q learning algorithm in the lower level to enable it to deal with high-dimensional complex tasks in a multi-agent environment in a decentralized manner. Finally, it introduced a skill discovery strategy on the basis of independent Q learning at the lower level, so that agents could learn complementary skills from each other. Compared the algorithm with the multi-agent reinforcement learning algorithms and the hierarchical reinforcement learning algorithms on the two simulation experimental platforms of simple team movement and StarCraft Ⅱ respectively. The experiment shows that the algorithm has improved performance indicators such as rewards and the victory rate of both sides, improves the decision-making ability and convergence speed of the entire multi-agent system, and verifies the feasibility of the algorithm.

[2023-03-03] Accepted Paper /doc/2022.12.0795/v1

[2023-09-05] Printed Article /doc/2022.12.0795/v2