In　this　paper,　a　reinforcement　learning-based　method　for　controlling　the　swarm　of　unmanned　aerial　vehicles　(UAVs)　in　complex　environments　with　static　and　dynamic　obstacles　has　been　presented.　The　method　in　this　paper　utilizes　double　deep　Q-network　(DDQN)　algorithm　to　train　the　action　of　virtual　leader　in　UAV　swarm,　which　enables　the　UAVs　to　effectively　navigate　through　complex　environments　while　avoiding　collisions.　Besides,　the　artificial　potential　field　method　is　also　applied　on　each　individual　UAV　to　deal　with　emergency.　The　effectiveness　of　the　proposed　approach　is　evaluated　by　simulations　in　complex　scenario.　©　2023　IEEE.