The　deployment　of　unmanned　system　swarm,　especially　unmanned　ground　vehicles　(UGVs)　and　unmanned　aerial　vehicles　(UAVs),　has　gained　significant　attention　in　various　applications.　However,　the　effective　navigation　of　unmanned　system　swarm　in　unknown　and　dynamic　environments　remains　a　formidable　task.　To　address　this　challenge,　a　novel　approach　that　combines　reinforcement　learning　and　artificial　potential　field　(APF)　methods　is　presented　in　this　article.　The　proposed　framework　utilizes　the　double　deep　Q-network　(DDQN)　with　a　prioritized　experience　replay　mechanism　to　train　the　actions　of　the　virtual　leader,　which　leads　to　the　motion　of　the　whole　unmanned　system　swarm.　In　addition,　the　APF　method　is　also　applied　to　each　unmanned　system　in　swarm　to　achieve　distributed　obstacle　avoidance.　By　training　virtual　leader　to　guide　all　individuals,　the　unmanned　system　swarm　can　adapt　and　navigate　safely　in　unfamiliar　surroundings.　Simulations　and　experiments　validate　the　proposed　method,　demonstrating　enhanced　obstacle　avoidance　and　autonomous　navigation　for　the　swarm.　　©　1963-2012　IEEE.