In　this　article,　an　evolution-guided　adaptive　dynamic　programming　(EGADP)　algorithm　is　developed　to　address　the　optimal　regulation　problems　for　the　nonlinear　systems.　In　the　traditional　adaptive　dynamic　programming　algorithms,　policy　improvement　is　typically　reliant　on　the　gradient　information,　according　to　the　first　order　necessity　condition.　However,　these　methods　encounter　limitations　when　calculating　the　gradient　information　becomes　infeasible　or　system　dynamics　is　not　differentiable.　In　response　to　this　challenge,　the　evolutionary　computation　is　harnessed　by　EGADP　to　search　for　a　superior　policy　during　policy　improvement.　Therefore,　compared　with　the　traditional　methods,　scenarios　that　gradient　information　is　unavailable　can　effectively　be　handled　by　EGADP.　Additionally,　the　convergence　of　the　algorithm　is　proven　to　enhance　the　rigorousness　of　the　developed　method.　Finally,　the　three　simulation　experiments　with　realistic　physical　backgrounds　are　conducted　to　comprehensively　demonstrate　the　effectiveness　of　the　established　method　from　different　perspectives.